Sunday, October 4, 2009

iPhone Video Output



There are a couple of options available when preparing to perform a demonstration of an iPhone app to a larger group of people. All these options have flaws, however. The "gather around" method, for example, doesn't work well for groups of more than a few people. Any more and you would ideally want to project the iPhone screen contents onto a big screen.

The "iPhone simulator method" is probably the most used method. It's simple to set up but limits your app navigation to the mouse, making multitouch gestures difficult to perform. Also, there is no way to show off accelerometer functionality like "shake to undo".
The "camcorder method" avoids the simulator's issues, but can be a hassle to set up. It also restricts your movement as you have to make sure to keep the device on screen and avoid reflections as best you can.

Doing it Steve's way
Why don't we take a page out of Steve Job's playbook and mirror the iPhone screen directly from the device onto the big screen?
Sadly this is not straight forward. So far only the iPod application supports video output via Apple's AV cable, at least that's what I thought until I stumbled upon Rob Terrell's iPhone App Video Mirroring blog post (go read it now, I'll wait).

Using the private MPTVOutWindow class in the MediaPlayer.framework API, Rob's code mirrors the iPhone screen onto the AV cable. For demonstration purposes this is a great solution, but being based on a private API it should not be left in the app when submitting it to the App Store.

Originally the code did not support on-the-fly orientation changes and touch indicators, both of which I needed. Thanks to Rob posting the source code I was able to implement these changes and have since submitted the updated code back to him, although I haven't heard back after doing so.

Get your hands on the code
In the spirit of sharing I have prepared a sample project, TvOutputSample, which shows you how to add video output to an Xcode project. This is the application shown in the video above, it should build and run out of the box, using iPhone SDK 3.0.

Download the code: TvOutSample.zip
[UPDATE: The project has moved to github]

I should mention that since the video mirroring is software based it affects application performance somewhat. On an original iPhone 10 fps works well, on an iPhone 3GS I have had no problems running at 20 fps. The fps setting is near the top of the UIApplication_TVOut.m file, do your own tests to see what works best for your app and hardware.

The current version (as of October 2009) of the code does not support OpenGL video output.

28 comments:

spencerlucas said...

great post. THanks for putting this together. I have implemented your class and code in my app, but it is not displaying--it may be the A/V cord I have, so I am going to buy another one. However, have you put this code in on the 3.1.2 SDK?
thanks

Liam Sluyter said...

Can this be installed as a stand alone utility, like TV-Out, on an iPhone? If so, how?

The Evil Boss said...

Liam: This is just a way to enable TV output for the specific app in which the code is embedded. System wide TV output is not provided.

Liam Sluyter said...

Thanks for replying.

Could it be embedded in a specific app? I'm thinking the Sky Mobile TV app just released. Live streaming sport on your widescreen TV would be pretty popular. Dare say people would pay for it.

The Evil Boss said...

You need to have the source code for the application you want to embed TV output in.
If you are the developer of Sky Mobile TV you can embed it, if not I don't see how it can be achieved.

Uli said...

Liam,
even if you were the developer, you could embed the code into your app, but then Apple's tool for detecting private API calls would probably prevent you from actually getting it into the app store.

So don't go complaining to the developer to get him to add this code. He can't. It's only suitable for testing builds and demos made by the developers.

The Evil Boss said...

Uli,
very true. Thanks for the clarification.

Eric said...

Boss,

I successfully implemented the code, however, I needed permanent landscape mode as we're demoing on an LCD TV permanently in "portrait" orientation to look like a giant iPhone, so I forced "LandscapeLeft" orientation with the "mirrorView", it worked great, HOWEVER, not sure how to scale to full screen, the size is still the same as if it were in portrait orientation. I believe it's in the "bounds" parameter but I am lost, can you help?

Thanks for the great code! Helping us demo our free app!

Eric

johnellisdm said...

any progress on making this work with opengl? is it even possible? i think, if you aren't working on it, i'll try and put it in there. where should i send the revised code to?

The Evil Boss said...

Eric,
I no longer have the AV cable necessary to test this, so I haven't been able to look into it. Maybe you can scale up the mirrorView a little? You may need to set its scale mode to scale up the image as well.
If you can figure out how to blow it up to full screen please post back.

johnellisdm,
Actually Rob has updated his post at http://www.touchcentric.com/blog/archives/3 to state that OpenGL does work.
I haven't tried it myself.

The Evil Boss said...

spencerlucas,
I only ever tested it using Apple's AV cable. I have not tried it on 3.1.2, since I'm still on 3.0 and no longer have the cable.

Eric said...

Thanks for Rob's link. I will see if there's any help there.

SPENCERLUCAS,

I have used THIS code on 3.1.2 (am using it now) and it seems to work well. Using it on 3GS with Apple's composite A/V cable.

Can anyone help explain how I can "scale" mirrorView?

Thanks!
Eric

The Evil Boss said...

Eric,
I managed to borrow an AV cable today to try to get the image to fill the screen when the device is held horizontally.

The following worked for me on a Widescreen PAL tv.

In the startTVOut method do the initialization with an 800 x 800 rectangle and set the scaling to AspectFit like so:

mirrorView = [[UIImageView alloc]
initWithFrame:CGRectMake(0, 0, 800, 800)];

mirrorView.contentMode =
UIViewContentModeScaleAspectFit;

After doing this the image fills my TV screen when the device is horizontal, of course now the top and bottom are cut off when in portrait mode.

Good luck, and please post a link to your app.

Techy said...

I added this to my app and i got the bottom right corner of thipod on the top left corner of the screen and the rest of the tv screen was all fuzzy

roov said...

Boss,

Any chance this code has been updated for the latest SDKs?

Reuven

The Evil Boss said...

roov,
I haven't tried it with iPhone OS 3.1.3 which is the current version. According to Eric's comment posted above it works well on 3.1.2 and since the difference between the two are minor, my guess is it works.

The upcoming iOS4 is expected to contain public API:s for output using the new VGA adapter, so this code should be redundant shortly.

roov said...

I actually tried it on 3.2 and I had some issues...

Re iOS 4 - I didn't hear any announcements on video-mirroring/video-out. Do you have any public pointers?

The Evil Boss said...

iPhone OS 3.2 is for iPad only, so I wouldn't expect the code to work for that out of the box.

In 3.2 there is already official support for video output, see External Display Support in the 3.2 release notes.

I expect those API:s to be present in iOS4 for iPhone as well.

roov said...

In 3.2 there is already official support for video output, see External Display Support in the 3.2 release notes.

I saw that - but couldn't find any sample code showing how to implement anything like what you had with a "TV Out" drop-in class.

Do you have any pointers?

The Evil Boss said...

Actually, yes I do have a pointer that might help you.

I just remembered that Matt Gemmell recently provided some sample code for iPad Video Out.

It's not a drop in screen mirroring solution, but may help you get the basics working. Check it out.

roov said...

I tried it with an iPhone w/o success :-(

I guess I'll have to wait until iOS 4 SDK is officially released and hope someone can update the sample code...

roov said...

now that iOS 4 was released, any chance of updating the sample to work for the formal SDK?

The Evil Boss said...

Yes, there is a chance I will do that. However, I'm currently focusing on getting some higher prio projects done.

I will also have to invest in an iOS 4 compatible phone before I can port it to iOS4, so don't expect it to be done any time soon.

Philip said...

Hey there Evil Boss. Would really appeciate if you could help getting it work with iOS4 (SDK4.0) as I have some demos to do. I cannot seem to find SDK 3.1.3 any more to downgrade to keep working!!

The Evil Boss said...

Native video mirroring available on iPad2 according to today's Apple presentation, hopefully coming to iPhone and first gen iPad as well.

It's about time really.

roov said...

Evil Boss said:
"Native video mirroring available on iPad2 according to today's Apple presentation, hopefully coming to iPhone"

Can anyone confirm that indeed video mirroring "just works" with the iPad 2?

How about the iPhone 4?

humpty said...

The Evil Boss may well deserve his name... after alot of looking it appears that the original code may have been written by Rob Terrell (it appears the evil boss replaced his credit) - Rob has updated his code to GitHub and now it is compliant with standard interfaces and works with iOS 4.0 - just trying it now...
https://github.com/robterrell/TVOutManager/blob/master/TVOutManager.m
just saying anyways

The Evil Boss said...

Humpty: You are right, the original code was indeed written by Rob Terrell. As is also noted in the article.

The source code file UIApplication_TVOut.m, which is the file which contains his code together with my modifications, retains Rob's credit in the header comment as well as information about the changes I've made.

Rob was informed about this code before it was published. He deserves, and has received, full credit for making this possible.