RealProducer Scripts

by Jamie Zawinski

From 2000 through 2009, we used to use RealVideo for our streaming video webcasts. Since 2009, we've switched over to a much simpler Flash-based stream.

Keeping the RealVideo server up and running was no trivial matter. I eventually came up with a set of scripts that made it operate itself mostly unattended.

But, by 2009, Flash-based video was high enough quality (and so much easier to use) that there was really no reason to keep beating my head against the RealVideo software.

Here is how video gets from us to you:

When we were still doing RealVideo, it went like this:

Here are the shell scripts that made all that work. The whole setup was pretty much held together with spit and bubblegum, but it worked pretty well for me.

Getting RealProducer to talk to RealServer through a firewall that is doing NAT is also a bit... challenging. I finally figured out how to do it, though, and here's how.

RealProducer communicates with RealServer over some or all of the following ports:

So at the least, your firewall has to allow those to pass between the encoder and the server.

But if you're doing NAT (Network Address Translation, meaning that the machines on the inside of the firewall have a different idea of what their IP addresses are than what the outside world sees) then life is harder.

The bottom line is, you can't use the Linux ipchains package if you want to use RealProducer through the firewall, no matter how many ports you open up. However, OpenBSD's ipf package works fine. (I have not tried ipfw.)

The reason is this: say you have three machines:

So firewall is doing 1:1 NAT for encoder, so that encoder itself thinks that it has the address ``172.28.1.5'', but the outside world thinks that it has the address ``166.88.23.5''.

If you do this on Linux, then the packets that arrive at server from encoder will have a source address of 166.88.23.1 (the external address of firewall.) If you use OpenBSD, they will have a source address of 166.88.23.5 (the external address of encoder.)

For most protocols, either way works; but for Real, only the latter works. I don't know why.

My only guess is that somehow, Real is encoding its vision of the address of the encoder machine in the protocol data itself, instead of getting it out of the TCP and UDP layer. So the firewall doesn't know how to reach into the data and rewrite those IPs. I can't imagine why the software would be doing this, but that's the only explanation I can come up with for the observed behavior.

Here's how you make it work:

On the firewall machine, set up the ethernet interface that is attached to the outside world to bind to multiple addresses in /etc/hostname.xl0 (or whichever it is): 

    inet 166.88.23.1 0xffffff00 NONE
    inet alias 166.88.23.5 0xffffffff NONE

Set up a bidirectional mapping in /etc/ipnat.rules: 

    bimap xl0 172.28.1.5/32 -> 166.88.23.5/32

In /etc/ipf.rules, let your encoder send packets out to the Real ports, and also get replies back (that's what ``keep state'' is): 

    pass out proto tcp from 172.28.1.5 to 209.133.35.83 port = 554  keep state
    pass out proto tcp from 172.28.1.5 to 209.133.35.83 port = 4040 keep state
    pass out proto udp from 172.28.1.5 to 209.133.35.83             keep state

And that's about it. I would rather have had all of my machines running the same operating system, for simplicity and ease of upgrades, but the only sensible thing was to use OpenBSD for my firewall, because the Linux firewall code was simply not up to the job.

<rant> I understand that the development Linux kernels have a brand new firewall package in them. Great, this is the third rewrite of the Linux firewall code in three years (with a third incompatible syntax, mind you.) It's as if whoever is responsible is treating this as a learning process or something. What they should do is just port the BSD code to Linux, since it's more powerful, older, widely used, and stable. But the free Unixes believe in nothing if not the doctrine of ``Not Invented Here.'' </rant>