Drew Scott Daniels' Blog

Thursday, January 1st, 2009 (2009/01/01)

Drew Scott Daniels

• Updated 2009/01/02, added links to build EVA8000 firmware.

Open Source on my TV

I recently got a new LG TV. It has an “Open Source Software Notice” with the caviet “This feature is not available for all models.” It notes that it may be running Linux kernel 2.6.12, busybox, uclibc, and Nanox. To get a CD (for a charge of distribution and media), within 3 years of distribution, I have to write to Opensource@lge.com which I think I’ll give a try. There’s a USB port and RS-232C in port on the back of the TV for “service only” which have the note “Used for software updates.”. I can’t find any other details so I guess it’s time to make some inquiries, and make sure I’ve got a nice null modem cable which I can use for lots of other projects.

I’d like to find a way to make use of the MPEG decoder that’s on my TV. A cheap encoder that I got the link for went for several hundered
dollars. I don’t know of any easily accessible digital television (DTV)
signals in my area. I’m wondering if Shaw offers free to air (FTA) DTV
that my TV will accept as CADTV (cable DTV?). I know PBS in Minnesota, and FOX in Pembina, ND may theoretically be accessible, but I don’t have time for that until it’s warm enough outside.

Since the TV has a nice “SVGA” input plug (that they call “RGB(PC)”) in
the form of a “D-sub 15 pin cable (VGA cable)”, I may try and hook up a computer to it. I’m conserned about the noise of the fans which seem to fail fast. I also would like to run another circut so I don’t have to stretch to the next circut’s plug. I’m thinking this could replace my Netgear EVA8000 – Digital Entertainer HD.

Netgear EVA8000 runs Linux

Netgear’s EVA8000 has a support page here, but the main forum and a restricted beta forum seem to have quite a bit of information not in their support site.

My Netgear EVA8000 runs Linux, and has several open ports on it. It
seems like the firmware is encrypted though. I can’t find anyone posting information like even a port scan so I may do that later. I’m worried that the Netgear EVA9000 will/has caused the developers to stop working on the EVA8000 which I hear ran out of memory.

Although for what they call GPL compliance, they offer to let you
download source code for the EVA8000, it seemed to be the original
version only (I later found the Feb 2008 release). It also seemed to be stripped of quite a bit of functionality. I also can’t easily figure out a way to upload the code which I think is intentional. I guessed it was a simple block device upload, but it didn’t mount in loopback. I should run testdisk on the file just in case. I think I remember reading somewhere about an encryption password being used to do updates, but it seems others have figured out how to dissect the image like this:

dd if=$FW of=crc bs=1 count=32
dd if=$FW of=bootkernel bs=1 skip=32 count=$((0x210000))
dd if=$FW of=jffsroot bs=1 skip=$((0x210020))

I just found a forum thread about how to edit the firmware images, and I’ve seen they’ve updated the source code to the latest non-beta release. Versions in between don’t seem to be available nor do beta versions. Unfortunately it seems you have to register to see this.

Another use for a null modem cable is connecting to the EVA8000 during boot and after interrupting the boot (by sending a few escape’s?), typing:

config cmd root=/dev/nfs ip=bootp
boot net

All at the boot prompts.

I see that it runs a 2.4 kernel which bothers me (2.4.22-uc0-sigma-20051018-nm). When I worked with embedded systems, I saw a few that updated 2.4 kernels to run with their reference boards as opposed to 2.6 kernels.

The EVA8000′s PC software called Digital Entertainer for Windows (DEW) seems to have at least one Windows audio driver which would be nice to learn how to use. I’m pretty sure that is what it is using to play iTunes music.

The DEW software comes with receiver.exe which seems to be what’s used to play things like YouTube videos. This makes me wonder if there are other videos that the DEW software manipulates (re-encodes, or redirects?) prior to going to the EVA8000.

Wireless network drivers

It’s a bit frustrating to see the software upgrade path for wireless
network devices stop. On old versions of Windows it’s easy to see that
the cost of maintaining the wireless network stacks for various
families of cards complicates things. On Linux, it’s easy to see the
problem is time and motivation. Now that there’s a standard wireless
stack on Windows Vista, and Linux, I hope that I can do things like
setup a Wireless Access Point, mesh network, etc. all with just a
software update and some configuration. I’ll have to be careful about
making sure my next devices work with the new standard wireless network stacks.

Mass storage

Now that I’m doing more with video, and I’m actively backing up many of the computers that I have, I’d like to get more storage. I evaluated
getting more drives, getting a Network Attached Storage (NAS) device
like the D-Link DNS-321, but for the cost, I think I should look at
online storage. I’ll likely still need a few extra drives, but not the
high capacity I was looking at.

I’ve decided that for home applications, I don’t need a RAID array. The
costs of maintaining the drives, the controller etc. is extremely high. I
seem to currently get all the relevant benefits with Bacula backups
plus the bonus of a long retention period.

Optical media (e.g. DVD’s) and other “removable” media seem to have a
very high time cost. They also require manual intervention on a regular
basis. Reloading any data requires finding the media and inserting it
which often doesn’t feel worth while for just verification.
Additionally, one should periodically do full backups and manage the
removal of old backups (AKA retention). Thus the removable media method of backup does not seem suitable for backups. Optical media seems less useful for movies too except that it’s so cheap.

I’ve been trying to think of the common modes of failure to help come up of ways of recovering from future failures. Most of my past failures
have been due to hardware problems, though a few seem like they may have been software triggered. Software for managing files like NTFS and ext3 seem to be a lot more reliable as are the OS’s that run the software. Hard drives are also more reliable with various SMART and related features. CD-R’s however have become less reliable from what I’ve read.

Power supply related failures causing hard drive failure has happened to me a few times. Drives failing to work after long periods of
inactivity… Most failures have resulted in the loss of an entire data
set.

Some of the online data storage have a 1GB file size limit. This is
probably to prevent the storage of large amounts of data. The easy work around for me is to set bacula to 1GB volumes.

Drew Scott Daniels’ resume: http://www.boxheap.net/ddaniels/resume.html
Originally from: http://www.boxheap.net/ddaniels/notes/20090101.html

Sunday, July 23rd, 2006

Drew Scott Daniels

I am maintaining a variety of old machines including many running
Windows 95. I have many licenses for Windows 95, but not as many for other versions of windows (although my MSDN subscription allows me to install certain versions of Windows on certain computers).

Applications I use on Windows 95 include:

• Abiword
• VideoLan Client (VLC)
• Opera Web Browser

Problems

I’ve had significant trouble with, Mozilla Firefox mostly due to its
installer. I believe Firefox would run fine, but it was taking too long
to install so I switched to Opera instead.

Abiword had a well documented required that I download some
“redistributable” Microsoft files. I was considering OpenOffice.org, but
it is quite bloated and has many features that I and other users do not
use. I use OpenOffice.org on several Windows XP machines quite happily.

One feature that I miss in OpenOffice.org’s Writer is a grammar checker. When I was in grade 10 I had my grammar level tested. The test indicated that I had a grade 14 level of understanding. Despite that, I enjoy grammar checkers such as the one in Abiword. A number of years ago I tried many word processors including Correl WordPerfect, Microsoft Office, Microsoft Works and Lotus WordPro. I found Lotus WordPro’s grammar checker to be so useful that I still like to run it over documents that I proof read. I hope to have more time to see if Abiword’s grammar checker is up to that level, and maybe help improve it.

VLC worked on one Windows 95 machine, but not another due to missing dlls. The missing dlls could probably all be downloaded, but I ran out of time. I installed two Microsoft packages after searching to find the source of the error messages that I was getting. A third error message left me without time. It had some kind of winsock dll requirement that could probably be removed if the dlls were delay loaded. I would imagine that it would be the same case for several other dlls that VLC required.

I find VLC to be one of the best video and audio players available.
It supports almost every format I want, is OpenSource, has support for
professional broadcast standards, and is very portable. I run VLC on
quite a few different operating systems including Windows 95, Windows
2000, Windows XP, Debian Linux and others. Unfortunately I currently
can’t say I run it on Apple’s MacOS X version 10.2 as it seems to crash. I believe the version 0.85 was the one that was crashing on startup. One other problem I have with VLC is that its playlist seems to cause the application to crash under certain circumstances. Rather than look into it, I’ve simply avoided the playlist feature.

I’m hoping to use VLC or ffmpeg to transcode some movies I have into
a format that I can watch and listen to on my 200MHz laptop. In the past I used to play 128Kbps mp3′s at lower quality on a 486 DX2 50Mhz. To do that I think I used WinAmp’s decode at quarter quality option. I think I can decrease the decoder requirements by lowering the frame rate. I was also considering lowering the bitrates of the video, audio and stream, but I’m uncertain as to whether that might mean more processing. Some compression/decompression algorithms take more system resources when trying to lower the bits per byte ratio (“better compression”).

Time has been a valuable resource lately. I finally got around to
upgrading my laptop to Windows 98. I spent quite a few hours trying a
few weeks ago, but the installer refused to work as I was trying to
upgrade with Toshiba’s Windows 95 winutils installed. I tried figuring
it out what was going on by reading all the documentation I could find
on the computer, and on the Win98 CD, but there was very few hints. One day, when I had some time on another computer, I searched and found that Toshiba has a nice upgrade web page. The trick to the upgrade was a separate uninstaller that was not linked to from the list of downloadable files.

My upgrade to Win 98 required a Linux boot CD, and a USB storage
device. The files I wanted/needed would not fit on a floppy
easily/conveniently. Win 95 does support USB, but it does not support
mass storage devices. That means that it would not support my USB key, any external USB drive, my digital camera… It also doesn’t support the Palm Pilot USB driver, and I couldn’t get a dlink USB wireless network key to work.

One of my motivations for updating, was that despite my many hours
working on getting my WPC11v4 working, I found that I required special out of tree Linux drivers, or Windows 98. I did find some indications that it might work on Windows 95, but it seems that was the WPC11v3 using a different chipset, the European version… The RealTek RTL8180 (or RTL8180L?) reference driver would install, and I believe attach, but not work.

More on these topics later.

Drew Scott Daniels’ resume: http://www.boxheap.net/ddaniels/resume.html

Originally from: http://www.boxheap.net/ddaniels/notes/20060723.html

Today on CBC’s “Up To Speed”, they had a business columnist talking about user interfaces. She complained that they shouldn’t need training to figure out how to use a photocopier. She further went on to talk about how the design of Research In Motion (RIM)’s BlackBerry keyboard went through quite a bit of research.

I’m a long time fan of Palm (formerly of 3Com). PalmOS, most Palm
Pilots and HandSpring portable data assistants (PDA’s) have a witting
interface called Graphiti. Grafiti can be used to write with a stylus
(pen like stick), onto a touch sensitive surface. The character language of graphiti is English writing characters, simplified for easier
recognition. This simplified language causes the users to have to learn
it if they want to take advantage of the high speed input. With the
advent of the Palm Treo, a BlackBerry like product, Palm looked at, and
later removed the graphiti capability. Until very recently, one could
download graphiti for the Treo, but without graphiti, stylus input
didn’t make much sense. So the company bowed to pressures and put a qwerty keyboard set of buttons at the bottom of the device. They also shrunk the touch pad area to make more room for this new keyboard.

With my arguments above, it might seem like Palm made a mistake in
adding a keyboard, but maybe not. Requiring a stylus makes one handed operation, difficult or impossible. Generally, this wouldn’t matter, except that the Palm Treo is intended to be used out in the field. In the filed is where people have to carry or hold other things. What though do people need one handed operation for? Primarily just for making calls, or acknowledging/checking incoming messages. If it’s
anything involved or time consuming, wouldn’t people want to free up
their other hand to pay more attention to the device? Of course if they
do that, then one step further is finding a flat surface at elbow hight.
Full size keyboards are by far a faster interface device. One could (and
probably still can), buy and external keyboard for Palm devices. My
personal preference is for Graphiti though, as if I can use a full size
keyboard, I can probably use a full size display, and… well I’d want
to use a laptop or notebook computer.

With laptops being less portable, and not needing one handed operation very often, my preference for Graphiti is strong. The real impractical ideal input would be Voice recognition. Unfortunately, that probably requires more processing power than would be available in a highly portable device (currently).

A month or two ago, at a consumer electronics show, it was announced that the new Palm Treo (the 700?) would run Windows CE. Besides the luggage that Microsoft’s Windows platforms have carried with them, I’m not aware of any version of Graphiti for Windows CE. It would be unfair for me to say that Windows CE likely requiring battery depleting hardware would be three strikes. Without the budget to buy a new Palm Treo, or use any of the extremely overpriced wireless data services for it, I probably won’t be researching which of those three strikes, if any aren’t true, or can be overlooked.

With doing research into real time operating systems, and cheap
alternatives to Windows XP, I’ve started to have a reason to look into
Windows CE. What’s really given me a peak into Windows CE is that
Microsoft Visual Studio .Net 2003 (and VC8 AKA MSVS 2005) shows help for CE library functions before any other results (and usually brings it up in such a way that one has to hunt to find the non-CE results).

From the Microsoft developer’s network (MSDN)’s help, I’ve found that
Windows CE supports many of the same functions as are available to most Windows applications. This means it might be easy to translate or port programs to the Windows CE platform. That’s not quite a home run, as there still is some work required for porting. Still the prospect of having many applications for the Windows CE platform might make it as popular as the PalmOS platform. When looking at the Treo 600, the salesperson commented to me that one of the reasons that people preferred the Treo’s over the Blackberry, was the number of applications for it. Certainly that has been a factor for people choosing Windows over MacOS or Linux in the past (now most Windows applications can be run on those platforms). I haven’t seen much of a developer community around the blackberry, but there may be one.

Windows CE might be called Windows Mobile now. Microsoft has split their products in many different ways these days, and renamed old products several times. Product names has been a problem for me in determining what it does, and how it’s different from other products. I’m a developer with a subscription to MSDN’s stuff, and I although I read and write Microsoft based things almost every work day, I still can’t keep track. I think however, I’m well ahead of the average user. The easiest way to learn about the products is to use them. Maybe someone might pick up the tab for me to get a new Palm Treo and give it a work out.

I’d like to write more about Voice Recognition, battery consumption,
and research the unknowns above, but I’ll save that for later.

Drew Daniels Resume: http://www.boxheap.net/ddaniels/resume.html

Originally from: http://www.boxheap.net/ddaniels/notes/20060309.txt

The differences between OnClose(), OnDestroy(), and DestroyWindow() (MFC/WinAPI stuff).

I knoticed one programmer uses OnClose() to try to prevent the user
from closing the window when they shouldn’t. To do this, he only called CDialog::OnClose() from his overloaded OnClose when closing was allowed.

OnDestroy() is another CWnd member. It’s called durring the
destruction of the CWnd object. This is probably the most useful
function for doing cleanup.

DestroyWindow() is yet another CWnd member. The MSDN documentation suggests that it might never get called unless the program explicity calls it. There is however a note that MFC’s might call it. Which, or when I’m not sure (isn’t CWnd part of MFC?).

Drew Scott Daniels

Resume: http://www.boxheap.net/ddaniels/resume.html

Originally from: http://www.boxheap.net/ddaniels/notes/20060102.html

The content here is written without the permission of Linear Systems
Ltd., but only uses publicly available information.

Some pages at http://www.linsys.ca

• Linear Systems Ltd.’s web site’s site map seems to make it easier to navigate the site. Google seemingly wasn’t indexing the site.
• Linear Systems seems to have quite a few different Broadcast Boards available. These look to be branded DVEO boards (many or all of the pictures show the word DVEO on the boards). There seems to be mostly boards for DVB-ASI, ATSC (SMPTE 310M), and SDI (SMPTE 259M).
• IP Caster is a pair of systems designed to receive ASI at one end, and then transmit ASI at the other. In between them is some kind of IP based network (probably using Gigabit Ethernet). The transmitter side of IP Caster looks like it might be able to be used to send to other appliances and software such as VideoLan.
• DVB Loop is a product that according to Computer Modules, uses Stream Valve software, and seems like it has a full duplex DVEO DVB-ASI PCI card in it.
• Quad Mux looks like it could be quite a nice, low cost, real time MPEG-2 transport stream multiplexer. The page says it supports MPEG SI tables such as PAT’s, PMT’s etc..
• TimeShifter looks like a low cost way to move content from one time zone to another, keeping prime time content, in prime time for the new time zone (e.g. a 9PM show in the eastern time zone could be played out an hour later to be 9PM in the next time zone over).
• A variety of software is available from Linear Systems.
• It seems that Linear Systems gives away their drivers, and some SDK’s for their cards at their downloads page.
• A variety of manuals are available for free download from Linear Systems’ Support -
  Manuals page. Of particular interest is the Master
  Technical Reference Manual which has quite a few details on the
  features of many of the DVEO boards, their operation, how to qualify a
  system, and even some introduction to DVB, MPEG and other broadcast industry concepts.
• The Support – Data Sheets page seems to contain both Data Sheets and a few white papers. Of particular interest is discussions on high speed (AKA throughput) data transfers, and information about overflows.

Drew Scott Daniels’ resume: http://www.boxheap.net/ddaniels/resume.html

Originally from: http://www.boxheap.net/ddaniels/notes/20051128.html

Issues of interest:
- Road traffic controls
- Road traffic problems (potholes, etc.)
- Fuel efficiency
A bug tracking system (BTS) for city work is a concept I've considered.
Citizens could then file "bugs" and various departments could manage
them. In the absence of the willingness of employees, volunteers could
do the work.

- Logistics of the transport of goods (could be more efficient)
Lots of empty trucks and train cars travel around on a regular basis.
Encouraging the use of local goods can decrease pollution, improve local
economies, make regions independent, and increase the quality of goods.
Local food and other perishables are generally in better condition if
they have to spend less time travelling.
Margins on goods from chain stores might be higher than the cost benefit
they gain through Economies of Scale.
- Regulation of household cleaners
In workplaces the Workplace Hazardis Material Information System (WHMIS)
requires that material data sheets be available for each cleaner or
hazardis material. On the sheet is a list of chemical ingredients, their
hazards, symptoms of hazards, and first aid.
- Airline regulations
On PBS I saw an episode of NOVA about an investigation into an airline
crash (Swiss Air 111?). In it they said that there were some
recommendations for industry regulations that were never implemented.
Some were as simple as requiring smoke detectors.
- Deregulating travel
Most airports say to arrive an hour early. While a large part of the
delay is due to the airlines, security delays are long. Security at
airports is a good idea, but the methods, purposes and other aspects
need to be evaluated. Not allowing a nail file through, but having other
such things for sale on the other side doesn't make much sense. Even if
the area past security were to be restricted, one should consider the
people sped through (e.g. workers and frequent travellers). Also other
items should be considered including the same kind of items as a prison
does (e.g. anything that can be made into a "shiv"). Even this much
restriction won't keep all possibilities of malicious intent "in check" as
employees in the industry must have access to dangerous things
including equipment for maintenance, fuelling, cargo handling, and the
air planes themselves must be kept available. As Bruce Schnider has said,
better security would be people trained to look for suspicious activity.
- Preventing insecure national ID cards
Bruce Schnider probably has a good number of articles talking about how
getting someone's ID doesn't prove that they're a criminal or not. It
may not even be a reliable way to get any statistics on a person unless
the people in charge of the ID have the proper policies in place to
insure the completeness, accuracy and integrity of such information is
insured.

Originally From: http://www.boxheap.net/ddaniels/notes/20050903.txt

One of the things I've been thinking about writing about is interesting
and common math problems. I'll start with some easy ones that I've had
to do at work (no trade secrets, just common math in the trade).

Notes:
- I'll use the prefix "0x" to denote that the number is written in hex
notation. E.g.'s 0x10 is 16, 0x1F is 31.
- There's 8 bits in a byte (I may not get to that until later problems
though).

Largest packet aligned buffer:
MPEG 2 transport streams have both 188 and 204 byte packets. In order to
transfer a high speed transport stream, large buffers are needed. Large
buffers reduce the number of interrupts per second, and make for
effective DMA transfers. To effectively process an MPEG-II transport
stream, buffers should be a multiple of the packet size. Splitting and
joining buffers is not only "a pain" to program, but also requires
otherwise unnecessary processing power.

With a maximum buffer size of 0x20000 (131072) bytes, and a chosen
packet size of 188, what is the largest packet aligned buffer size
allowed?

The answer is simply 131072/188*188 when the division is integer
division (i.e. no decimal places). To do this with a calculator that
only does regular division, one simply needs to remember the integer
part of the division. In this case 131072/188=697..... So, then I just
clear the calculator, and type 697*188 and get 131036 (0x1FFDC).

...

With a maximum buffer size of 0x20000 (131072) bytes, and a chosen
packet size of 204, what is the largest packet aligned buffer size
allowed?

The answer is 131072/204*204 where the division is integer division. The
final number is then 130968 (0x1FF98).

What is the minimum buffer size that is divisible by both 188 and 204?

To solve this, take out the common factors, and multiply the remaining
together. The factors in 188 are: 2, 2, 47. The factors in 204 are: 2,
2, 3, 17. So the answer is 2*2*47*3*17=9588 (0x2574), or
188*204/2/2=9588 (0x2574).

With a maximum buffer size of 0x20000 (131072) bytes, and a packet size
that can only be 188 or 204, what is the largest packet aligned buffer
allowed?

Using the knowledge from above that 9588 bytes is the smallest multiple
of both 188 and 204, it's simply 131072/9588*9588 (again where the
division is integer division). So the answer is 124644 (0x1E6E4).

Other numbers of interest:
- 196=188+8(64 bits is 8 bytes) for a 188 byte packet with a 64 bit
timestamp.
- 212=204+8 for a 204 byte packet with a 64 bit timestamp.
- 512 bytes is a common write size divisor for hard drives.
- 192=188+4(32 bits is 4 bytes) for the MPEG stride (MPEG 2 transport
stride?) format (HDV).
- 4 bytes (32 bits) seems to be a preferred DMA number (the PCI bus is
always at least 32 bits wide).
- 8 bytes (64 bits) might be preferred for certain DMA.

I'll probably write myself up a quick reference. When programming, you
can have the program calculate the right numbers given any buffer size,
packet size or other factor.
Originally from: http://www.boxheap.net/ddaniels/notes/20050820.txt

So today I got a new USB key. It's a Kingston Data Traveller. I decided
that I should set it up so that I can use it with the laptop I use on a
regular basis (a very old Toshiba Satellite 310CDS that's rattling). My
first step was to consider the file system format. I was surprised to
find that I couldn't format the device in NTFS format. I left it
formated at the default FAT32 and decided to get on to other things.

At home tonight I spent some time downloading the drivers for the device
and attempted to install them. The driver installer is an installshield
created one that's been winziped into a self extracting file (sometimes
called an sfx). Three layers of compression and installer junk managed
to make the under 49KiB of driver files take over 1MB. The second
frustrating thing I ran into is the installer is designed to detect the
operating system, and refuse to install if it doesn't think it'll work.
Well, I guess before that I had read a faq from Kingston saying that
Windows 95 doesn't support USB drives at all.

Before I get ahead of myself again, I'll go back and say that I tried to
do some research on what Windows 95 supports in the way of USB drives. I
didn't manage to find much, but I did find the usual indications that
earlier versions of Win95 didn't have any USB support, or that it wasn't
working. I already knew that USB support worked on my computer.

I guess I should have seen things coming ahead of time. In December I
had looked at trying to get pictures off a Fijitsu FinePix digital
camera. It too had an annoying installer, and claimed not to work with
Win 95. It further had a bunch of software bundled with it's
installation that I still haven't bothered to figure out. Luckily, the
installer for the driver itself wasn't hard to find, and I managed to
get the device driver, and some of the software installed.

Despite getting things installed for the FinPix camera, the software
complained about a missing dll function, and the driver didn't seem to be
working. I decided that with my many licences of Windows, I should try
to upgrade certain dll's with versions from newer versions of Microsoft
Windows. My results were of course that some of the important dll's
could not be replaced.

That got me thinking again about getting open source replacements for
certain components. I looked for a while, and decided that without a
better understanding, I might end up accidentally installing a dll that
needs an Linux shared library (.so) or something. My following of the
Wine Weekly News (WWN) on http://www.winehq.com and reading the ReactOS
developers/kernel mailing list indicated that some dll's from these
projects were defiantly dependent on components that I'm not ready to
replace.

So more recently (getting back to the USB key), I did another search on
the subject of replacing Microsoft Windows 95 dll's with OpenSource
compatible versions. I'm also now considering replacing the kernel and
other core files. I did find that WWN shows that they've been building
PE versions of their dll's for Win32, but it's not clear which can
replace the dll's in Windows 95. I get the impression that files from
ReactOS might be a better replacement than Wine's as they'll have less
Linux, BSD, Solaris related stuff in them and be created with binary
compatibility in mind for even more core pieces (e.g. no required
wineserver).

To date I've had no luck with either the USB key or digital camera under
Windows 95. I've decided that in order to start replacing Win95 on this
notebook, I'd better get a better understanding of the dependencies and
compatibilities of different components. To do this I'd like to get or
create a list of files, a graph (tree?) of the dependencies between
files, and a fresh compatibility status of the files from whatever
source I choose. Unfortunately ReactOS's compatibility page doesn't jump
out at me in searches (I remember seeing it once or twice). I also
believe both ReactOS and Wine don't list their compatibility in relation
to Windows 95, but to whatever the latest version of the component is.

So the processes I'll probably want to take will start with listing the
operating system files on the computer I'm targeting. Then I'll probably
use something like dependency walker (depends.exe from systeminternals?)
to figure out the dependencies of each files (as best I can). Then I'll
look at the compatibility status on the web. Last I may have to look at
the exports from both files. Since no one else seems to have published
this information, I'll probably write up my findings as I go. I might
even make it easier to install Open Source replacement components for
other versions of Windows by performing the same process using fresh
installs of other versions.

It's getting late now and I'm getting tired. I was planning to also
write about how to use unshield and winzip to extract files from
annoying installers. I also felt the need several times to explain why I
wanted open source replacement files, and didn't upgrade Windows
(remember I do have licences to newer versions). I guess I can quickly
say that I like having free access to the source of what I'm using so
that I or just about any other programmer can enhance/fix it. I also
don't want to install Windows98 or later on this laptop because it may
take more system resources, not run, and well I'd rather maximize the
use of my Windows 95 licences before using other ones. I've tried
ReactOS and Wine, and I know they're still not 100% replacements for
Windows (although extremely close nowadays). I also believe that other
people share my viewpoints and/or situations.

Maybe later this week I'll write more on the topic of replacing windows
components or Windows Device Drivers (wdm, ndis, inf, the wonderful
dpinst.exe and more), but for now it's time for me to get some sleep...

Originally from: http://www.boxheap.net/ddaniels/notes/20050817.txt

I was planning on writing about the problems I faced at work looking up
open source software for SMPTE 125M convertion. I kept finding SMPTE
timecode stuff (for MIDI), and other usages of the acronym SMPTE without
reference to which standard was being used. The one's related to SMPTE
125M are SMPTE 292M (HD-SDI), SMPTE 259M (transport of SDI and SDTI),
SMPTE 305M (sometimes called SMPTE 305.2M which is SDTI), and the
document on ancellary data. Actually SDTI really is quite different from
SDI except that it goes over 259M.

Anways, tonight I think I write a bit about linking and google. Yes,
part of the reason that I'm writing these notes is to increase the
ranking that I'll get for topics that I'd like employers to see. The
bigger way that I plan to get a good ranking is something I've
accidentaly found before. I've put a one line signature in my e-mails to
mailing lists with my resume's URL. I was hoping I could find someone on
the mailing list that might be interested, or might refer me to someone,
but instead I found that the html mailing list archives looked to be
increasing the rank of my resume. I guess this was a neat trick that can
work on google, and maybe on other search engine's that look at what's
linking to a page to give it a score.

When I finnaly am happy with the testing scripts that I'm working on for
my tarball enhancements I'll post the results to various mailing lists
that are development forums for projects with large tarballs (e.g. the
lkml, some kind of gimp mailing list, maybe some OpenOffice.org AKA OOo
mailing lists...). I've got my resume's URL in the scripts themselves,
but I also plan to put my resume URL tagline in my messages.

One of my problems with my tarball enhancement postings is that I'll
want a perminate place with my domain name that I can host the scripts,
but I'm getting free hosting from a friend (thanks Dean). I don't want
to generate a lot of hits on my friend's server due to the fact he
likely has better uses for his bandwith, and his ISP may not apreciate
it. To prevent such a load on the link to his server (and his server), I
plan to keep the scripts only on the mailing lists (archived in their
archives) until interest drops down a bit. I figure a few weeks would
do, but I'll probably wait a few months.

I'm really quite kean to get my scripts out the door, but I feel they're
not yet ready to stand up to the kind of critism that one gets on the
Linux Kernel Mailing List (lkml). I've got a script to do the actual
tarball creation, and one to show the difference between a normaly
generated one, and the one my script makes, but I don't have something
showing the amount of time that it takes. Measuring the sorting isn't
easy, as it's a series of piped commands. My shell scripting really
isn't put to enough use for me to be able to quickly work around such a
problem. I've checked a few howto's like the bash one, I've asked in the
bash scripting IRC channel, but I couldn't find an answer. I decided to
put the commands into a separate script and time that whole script.

The other problem I've run into is testing. My home computer was taking
a beating compressing and untaring etc.. I decided to use my
SourceForge compile farm shell to do the testing, but it's a pain to put
files onto them. It took me a while before I figured out I had to
download the files to my computer, and then upload them to the compile
farm's central server via sftp or scp. That's something I can do, but it
really compounds another problem I'm having. It takes me a while to make
progress on my free time coding projects, so new target files are
comming out for me to test. I want to be able to post on the lkml the
results of recompressing the latest 2.6 and 2.4 kernels. I keep
optimistically downloading the latest kernels and then having real life
interupt things long enough for me to need a new version to continue.
I'll stop doing that for a while though until I've actually got a draft
sitting in my posponed box of an e-mail to the lkml with the scripts
already finnished and attached or actually inline I think. That's
another problem. The lkml only accepts certain posts, and Linus only
usually accepts things that are in a certain format (plain text inline
iirc). That put me on a tangent of looking up the mailing list rules,
and reading the Linux Weekly News. It'll likely do the same once I get
close enough again.

So with all my knowledge, reading, and interest in digging deep into
open source stories that I see writen/posted, I've thought about trying
to get payed to write. These notes are a bad example of my ability to
write, but a good example of what I enjoy writing about. I've been
solicited once to write a book on Intrusion Detection from a genuine
publisher, but I kind of "fubbed" my responce. I said that I'd be
interested in contributing, but I didn't think I'd have time to write a
whole book. I kind of regret doing that, but I think it was the right
thing to say (just look at my bad record finding time to do coding). I'm
hoping however that a paying gig would actually let me take some time
away from real life to actually get things done (and I'm sure it would).
Of course I've got to stike a balance to keep my home life happy and
healthy (fammily, friends, and my own condition). I've offered to write
a peice on the history of the BSD's to the Linux Weekly News, but they
didn't seem interested. They do post BSD articles, and I was pitching
that I could write one that would show the parallels between AT&T vs The
Regents of Berkly (BSD) and the current SCO vs IBM etc.. It's
interesting how the history repeats itself. For good reference I'd
suggest reading the FreeBSD mailing list archives (a google search found
some good stuff).

Later I might publish the research that I used as part of my pitch for
my BSD history repeats itself story. I'm also probably going to consider
writng about why I don't want to publish my unrealized ideas. I'll also
probably talk about:
- Why I don't write about office politics
- Why I don't write much about my personal private home life (well,
maybe I made that clear <g>)
- My music idea's
- My thoughts and research into a self powered home (well actually
getting power form alternate sorces like sun, wind, water...)
- Thoughts on using "image stacking" for ameture (and hopefully
professional) astronomy (I'll talk about this because other people have
already implemented some of this)
- Some idea's for how people can generate data that's easier to compress
(e.g.'s typing in lower case when there's the option, removing obvious
redundant information, using the same words...)
- Perhaps my ideas on natural language processing
...

I may eventually post my project ideas from the last fourteen years that
I've been writing on paper.

Consider sending me money! My resume is at
http://www.boxheap.net/ddaniels/resume.html

Oh, and I'll probably write about resume creation and open source tools
to do it (hey, maybe lwn.net would be interested in buying that
article).

Originally from: http://www.boxheap.net/ddaniels/notes/20050811.txt

There doesn't appear to be any adopted standards for MPEG over IP. IP
over MPEG looks more interesting. Just packetize an IP stream into a
packetized elementary stream (PES) and multiplex it into a valid MPEG2
Transport Stream. MPEG-2 typically gets transfered over DVB-ASI, DVB-C,
DVB-SI, DVB-T and other protocols (even "ATSC" AKA SMPTE 310M).

So how do you packetize IP packets to go into an MPEG stream? Well that
depends on the source. I'd like to think that any IP source "worth it's
salt", is from a live network. Thus a network feed would need to input
into the packetizer, multiplex it and put it out over a different type
of device. I've heard of some people making a network device driver for
DVB-ASI cards, but at least one engineer I talked to said there's
probably a better way. He suggested keeping the regular characteristics
of the ASI device, and doing the packetizing in application space. I
managed to convince him however that the conveniences of creating a
network device which can be bridged would be far better. He stuck with
the separate device driver idea however and suggested one driver could
use the other.

So then the question is, how do you create a network device driver
that's just a packetizer, multiplexer and forwarder? No doubt there's
some good examples out there, and NDIS should make it easier. I still
worry about doing more than elementary processing in a driver might
cause some strange system behavior. I guess I should also say there's
probably an even easier way to do things in Linux and FreeBSD variants,
but I'm mostly focused on the Microsoft world as that's what I'm told by
Marketing is what's wanted.

On the opposite end you need a depacketizer? Or something to demultiplex
the stream, and put IP back out onto the network. I've seen this done in
software, and that might make more sense on this side of transfers. The
engineer that I speak of above however suggested that the unidirectional
nature of MPEG II transport streams would give another problem,
associating one direction of traffic with the other.

I'm not quite sure how other people bind one transfer direction with
another, but I remember several satellite companies offering service
that beamed high speed broadband internet access to customers and
accepted data back to them via telephone modem. So schemes to put two
different directions from seperate devices have been around for a while.
I just hope that modern network stacks are smart enough to remember that
it's allowed.

I remember someone telling me that the ARPA network was an experiment
designed with the goal that it be able to stay up, even if one link in
the network went down. It failed, or at least that's the punchline. The
modern Internet can't reroute if there's a failure in a router. There
was a fire in a telecom building in Toronto, and connections from
Manitoba Telephone Services (MTS) to Shaw in Winnipeg went down. I've
also seen where an outage in Shaw's network caused places to become
inaccessable, but if you had a proxy on a CA network accessable address,
you could access the rest of the internet. Those are just two local
examples that I know about. The CA network thing is political (I'm told
they're not allowed to carry commercial data due to their funding
grants). For what it's worth, I've also seen many shares of
misconfigured routers, the more obvious cases were with major telecom.
companies.

So back to MPEG transport streams. I know companies like Norsat have
been selling "solutions" to do these things for years, so I think
there's a market. Identifying the market potential is difficult for this
because it's not something most broadcasters, stations, and local
distributors are looking for. It's also not something that's really even
remotely accessable to consumers.

A similar issue that I've thought about for even more years is multiple
links between computers to increase throughput. I know lots of other
people have looked at bridging and bonding, but I wanted to look at it
at an even more insane level, serial ports. Actually I wanted to look at
paralell ports, modems, ethernet, etc.. I suppose it is possible to bond
all these links together, but it certainly isn't common enough that it's
as easy as listing the links (at least as far as I know).

So why bother with all this legacy stuff? Why not build a new network
card that can communicate at the full buss speed? Well actually we're
pretty close to that now. From my own experiences I've calculated that
modern HD-SDI cards must be close to maxing out the bus throughput.
I've also learned that multiple cards on the same bus can't allow a
faster network connection as of course there's only the one bus. Of
course I've seen computers with multiple bus's, but it's hard to know if
they're truly independent, or if they're more likely bridged. Even a
bridged network of bus's can allow each bus to operate almost
independently, if the parent bus is faster than it's children combined
there may be an advantage to using multiple cards.

Even further it's important to note that most modern bus's have
bottlenecks. When was the last time you looked up the DMA latency of the
motherboard you wanted to buy? I'll wager never. I've thought about how
useful this could be to consumers and whether there was a way I could
get the comany that I'm working for to publish regular results of DMA
throughput and latency. We could then get free motherboards. The idea
likely wouldn't work though as that's not really what the business
does.

Other crazy idea's I've had include using every processor on the system
to do computations including the IDE/ATA hard drives (they have RAM
too!). Alas of course most of it would be very convoluted to figure out
a way to use.

The more recent idea that I've had (since Mark Nelson's "random" binary
file challange was posted), was to figure out a list of common
instructions and library calls for which I could get more output that
would be required to issue the request for data (e.g. more bits from
register results than from instruction cost...). This idea has some
potential, but my current "hurdle" is finding the time to get and go
through a list of CPU instructions. Getting a list of available function
calls is also a challange although maybe a nice program to do it already
exists (just get the exports from all dll's etc?).

So as you might be beginning to see, one of my primary interests is
data compression. I used to be interested in the pure pattern finding,
and making the smallest representation possible of common data, but then
I started working in multimedia. It became obvious very fast that the
speed of compression actually is important (not just to those who can't
wait). If things don't compress fast enough you can get overruns, data
loss and ultimatly data corruption (even if that just means missing
bits/bytes/frames...).

One of my past pet projects was zlib compression of SDI (see StreamBed's
deflate option). I've found that I can gzip at 270,000,000 bps (that's
270Mbps in SI notation). The small b means bits of course. The problem
is that it ether needs a fast processor, or a simple pattern (like
colour bars). It may even need both. I haven't had the time to check.
Unfortunaly without the inflate option in StreamBed, customers aren't
too interested yet, and without customer interest my boss isn't too
interested yet either.

I later plan to talk about:

compression of MPEG transport streams (they're already MPEG
compressed, but the tables are text, and there's that predictable 0x47
once per packet or 188/204 bytes).
lossy MPEG table compression (recreate it to meet spec)
Alternate SDI compression
Compression in firmware
My literly magic tarball scritps to improve general tar.gz and
tar.bz2
compression with an order(1)+small sort preprocessor
My extensive prediction by partial match (PPM) research and
documentation (I've spent at least the last 5 years working on a new
algorithm that I have high hopes for, but strange results)
My ideas (hopefully implemented soon) for a very simple honeypot
like intrusion detection system
problems with current network security tools
Perhaps products and development directly related to my jobs


Drew Scott Daniels' Resume
Originally from: http://www.boxheap.net/ddaniels/notes/20050810.txt