{"id":14,"date":"2005-08-10T20:00:45","date_gmt":"2005-08-11T03:00:45","guid":{"rendered":"http:\/\/www.boxheap.net\/ddaniels\/blog\/?p=14"},"modified":"2009-02-07T12:54:45","modified_gmt":"2009-02-07T19:54:45","slug":"14","status":"publish","type":"post","link":"https:\/\/www.boxheap.net\/ddaniels\/blog\/2005\/08\/14\/","title":{"rendered":"20050810"},"content":{"rendered":"
There doesn't appear to be any adopted standards for MPEG over IP. IP\r\nover MPEG looks more interesting. Just packetize an IP stream into a\r\npacketized elementary stream (PES) and multiplex it into a valid MPEG2\r\nTransport Stream. MPEG-2 typically gets transfered over DVB-ASI, DVB-C,\r\nDVB-SI, DVB-T and other protocols (even \"ATSC\" AKA SMPTE 310M).\r\n\r\nSo how do you packetize IP packets to go into an MPEG stream? Well that\r\ndepends on the source. I'd like to think that any IP source \"worth it's\r\nsalt\", is from a live network. Thus a network feed would need to input\r\ninto the packetizer, multiplex it and put it out over a different type\r\nof device. I've heard of some people making a network device driver for\r\nDVB-ASI cards, but at least one engineer I talked to said there's\r\nprobably a better way. He suggested keeping the regular characteristics\r\nof the ASI device, and doing the packetizing in application space. I\r\nmanaged to convince him however that the conveniences of creating a\r\nnetwork device which can be bridged would be far better. He stuck with\r\nthe separate device driver idea however and suggested one driver could\r\nuse the other.\r\n\r\nSo then the question is, how do you create a network device driver\r\nthat's just a packetizer, multiplexer and forwarder? No doubt there's\r\nsome good examples out there, and NDIS should make it easier. I still\r\nworry about doing more than elementary processing in a driver might\r\ncause some strange system behavior. I guess I should also say there's\r\nprobably an even easier way to do things in Linux and FreeBSD variants,\r\nbut I'm mostly focused on the Microsoft world as that's what I'm told by\r\nMarketing is what's wanted.\r\n\r\nOn the opposite end you need a depacketizer? Or something to demultiplex\r\nthe stream, and put IP back out onto the network. I've seen this done in\r\nsoftware, and that might make more sense on this side of transfers. The\r\nengineer that I speak of above however suggested that the unidirectional\r\nnature of MPEG II transport streams would give another problem,\r\nassociating one direction of traffic with the other.\r\n\r\nI'm not quite sure how other people bind one transfer direction with\r\nanother, but I remember several satellite companies offering service\r\nthat beamed high speed broadband internet access to customers and\r\naccepted data back to them via telephone modem. So schemes to put two\r\ndifferent directions from seperate devices have been around for a while.\r\nI just hope that modern network stacks are smart enough to remember that\r\nit's allowed.\r\n\r\nI remember someone telling me that the ARPA network was an experiment\r\ndesigned with the goal that it be able to stay up, even if one link in\r\nthe network went down. It failed, or at least that's the punchline. The\r\nmodern Internet can't reroute if there's a failure in a router. There\r\nwas a fire in a telecom building in Toronto, and connections from\r\nManitoba Telephone Services (MTS) to Shaw in Winnipeg went down. I've\r\nalso seen where an outage in Shaw's network caused places to become\r\ninaccessable, but if you had a proxy on a CA network accessable address,\r\nyou could access the rest of the internet. Those are just two local\r\nexamples that I know about. The CA network thing is political (I'm told\r\nthey're not allowed to carry commercial data due to their funding\r\ngrants). For what it's worth, I've also seen many shares of\r\nmisconfigured routers, the more obvious cases were with major telecom.\r\ncompanies.\r\n\r\nSo back to MPEG transport streams. I know companies like Norsat have\r\nbeen selling \"solutions\" to do these things for years, so I think\r\nthere's a market. Identifying the market potential is difficult for this\r\nbecause it's not something most broadcasters, stations, and local\r\ndistributors are looking for. It's also not something that's really even\r\nremotely accessable to consumers.\r\n\r\nA similar issue that I've thought about for even more years is multiple\r\nlinks between computers to increase throughput. I know lots of other\r\npeople have looked at bridging and bonding, but I wanted to look at it\r\nat an even more insane level, serial ports. Actually I wanted to look at\r\nparalell ports, modems, ethernet, etc.. I suppose it is possible to bond\r\nall these links together, but it certainly isn't common enough that it's\r\nas easy as listing the links (at least as far as I know).\r\n\r\nSo why bother with all this legacy stuff? Why not build a new network\r\ncard that can communicate at the full buss speed? Well actually we're\r\npretty close to that now. From my own experiences I've calculated that\r\nmodern HD-SDI cards must be close to maxing out the bus throughput.\r\nI've also learned that multiple cards on the same bus can't allow a\r\nfaster network connection as of course there's only the one bus. Of\r\ncourse I've seen computers with multiple bus's, but it's hard to know if\r\nthey're truly independent, or if they're more likely bridged. Even a\r\nbridged network of bus's can allow each bus to operate almost\r\nindependently, if the parent bus is faster than it's children combined\r\nthere may be an advantage to using multiple cards.\r\n\r\nEven further it's important to note that most modern bus's have\r\nbottlenecks. When was the last time you looked up the DMA latency of the\r\nmotherboard you wanted to buy? I'll wager never. I've thought about how\r\nuseful this could be to consumers and whether there was a way I could\r\nget the comany that I'm working for to publish regular results of DMA\r\nthroughput and latency. We could then get free motherboards. The idea\r\nlikely wouldn't work though as that's not really what the business\r\ndoes.\r\n\r\nOther crazy idea's I've had include using every processor on the system\r\nto do computations including the IDE\/ATA hard drives (they have RAM\r\ntoo!). Alas of course most of it would be very convoluted to figure out\r\na way to use.\r\n\r\nThe more recent idea that I've had (since Mark Nelson's \"random\" binary\r\nfile challange was posted), was to figure out a list of common\r\ninstructions and library calls for which I could get more output that\r\nwould be required to issue the request for data (e.g. more bits from\r\nregister results than from instruction cost...). This idea has some\r\npotential, but my current \"hurdle\" is finding the time to get and go\r\nthrough a list of CPU instructions. Getting a list of available function\r\ncalls is also a challange although maybe a nice program to do it already\r\nexists (just get the exports from all dll's etc?).\r\n\r\nSo as you might be beginning to see, one of my primary interests is\r\ndata compression. I used to be interested in the pure pattern finding,\r\nand making the smallest representation possible of common data, but then\r\nI started working in multimedia. It became obvious very fast that the\r\nspeed of compression actually is important (not just to those who can't\r\nwait). If things don't compress fast enough you can get overruns, data\r\nloss and ultimatly data corruption (even if that just means missing\r\nbits\/bytes\/frames...).\r\n\r\nOne of my past pet projects was zlib compression of SDI (see StreamBed's\r\ndeflate option). I've found that I can gzip at 270,000,000 bps (that's\r\n270Mbps in SI notation). The small b means bits of course. The problem\r\nis that it ether needs a fast processor, or a simple pattern (like\r\ncolour bars). It may even need both. I haven't had the time to check.\r\nUnfortunaly without the inflate option in StreamBed, customers aren't\r\ntoo interested yet, and without customer interest my boss isn't too\r\ninterested yet either.\r\n\r\nI later plan to talk about:\r\n