May 08, 2004
XUL + Flash = Cross Platform Projectors
The aspect of Flash development that has intrigued me the most is without a doubt 3rd party projectors. Programs like SWF Studio and Screenweaver inspired me, and I set off to see if I could create that kind of functionality on my own. Thus, SharpFlash was born.
However, I started losing interest in SharpFlash. The "Windows-only" aspect of it really bothered me, and I didn't want to continue to devote time to something that left out a large portion of the Flash population. As much as I enjoy coding in C#, I hated to leave my Mac friends behind.
It seems that all 3rd party projectors rely on the Flash Active-X control to embed Flash content into a native Windows container. Because of the reliance on the Active-X control, our target audience is limited to Windows right from the start. I thought I'd have better luck with a Java solution (and SWT Flash), but I encountered the same Windows only fate. Java is cross platform; the Flash Active-X control is not.
That got me thinking.... How does Mozilla do it? Mozilla browsers have a Flash plugin that works on all platforms. Why hasn't there been a solution yet that piggybacks off of this functionality? Finally, there is.
Enter XUL.
You've probably heard a lot about XUL lately. I'll spare you the details about what XUL is, that's what google is for.
With XUL, we have the ability to embed HTML into applications that run on top of the Gecko Runtime Environment (GRE). The GRE is similar to the Java Virtual Machine in that you can write code once, and run it on any platform. .NET comes close, but there is no .NET framework that will run on a Mac or Linux (however, the mono project released beta 1 not too long ago).
So, what does this have to do with Flash and cross platform projectors?
By leveraging the GRE (and specifically, XPCOM), we can use the Mozilla Flash plug-in and run Flash applications cross platform.
To illustrate this concept, I've created a simple Flash text editor. You launch the application, click "read" to open a file, make some changes, and click "save" to commit those changes to disk. To see that the save actually works, clear out the text area, then read in the file again and watch it bring the saved contents in.
You can download the example here.
In order to run the application, you need a Mozilla-based browser installed. To run the application, just launch the .xul file with either Mozilla or FireFox. I tried launching the .xul file with FireFox but got a "c is not a registered protocol" error. The workaround for me was to open FireFox, then drag the .xul from Windows explorer onto the browser window.
NOTE: I want to stress that I'm still very early in the research project for this. This example is just a proof of concept, showing that something like this actually is possible. In the future I'll be trying to explore all of the possibilities. I'd like to add in an ActionScript API, and some way to bundle everything as an executable, but before I do I need to make sure that following this road is going to be worthwhile. I know you can launch the applicatiuon by creating a .bat file that launches the .xul application right from Mozilla via command line arguments, but I'm not sure about creating a standalone .exe that has everything bundled into it yet.
Also, I have to admit that I didn't actually test this on a Mac, or on Linux. I'm just assuming that it will work, and if it doesn't then I blame Mozilla for false advertising. :-P
Now, I'm going to attempt to anticipate some questions, and provide answers for them before hand. Read on for those...
Question: This all well and good, but I don't want to run a Flash application in a browser window! Is there anything else I can do?
Answer: My example is just a proof of concept. Right now, the only thing is does is local file access.
To be honest, I haven't explored all of the possibilities yet. With XUL, we can remove most of the browser elements and just be left with a window, a title bar, and the Flash movie - user's won't even know the Flash movie is running "inside of a browser" because we can get it to behave just like a desktop application.
Question: What about users that don't have Mozilla installed?
Answer: The GRE is more or less a virtual machine. It can be bundled into an application and installed if the user does not already have it. The GRE is a necessity to make the cross platform support work. You can't run a Java application without the JVM installed, and the same is true for cross platform Flash solution I demonstrated here.
Question: Why do we need to do all of this when Central is going to support local file access?
Answer: I'm not a big fan of Central. I don't like that my Flash applications are stuck in a shell with other Flash applications. I would prefer to develop applications that can stand on their own, or at least fake it really well.
You can probably use Central if all you want is file access, but my personal preference leads me to find a different solution.
Keep in mind, this example is just a proof of concept. There are probably things that I'll be able to do in the future with XUL/XPCOM that will supercede Central's functionality. In my example I just used local file access because that seems to be the most requested feature. Again, to be honest, I don't know what the limits of this initial research are, and I don't know what all is possible yet.
Question: Why embed Flash at all? XUL can do this all on it's own!
Answer I know it can. However, again, this is just a proof of concept. Think about creating a Flash game and having it save the game data locally.. that would have been a better example, I think.
Question: Shared objects let me do all of this already. What's the point?
Answer: Yes, shared objects let you store local data, but they're limited to the amount of storage the user specifices. Plus, with a shared object, you have no way of opening an arbitrary file on the user's system. The "open file" dialogue that pops up when the "read" button is clicked should be enough to demonstrate that this technique is more powerful than just using shared objects.
Comments
Thats pretty cool, excellent idea.
It works fairly well, although firefox did ask for permission before opening the browse dialog, and once i was able to bring in a txt file and edit it saving it out messed up the formating... but it worked!
Also I should mention that Northcode (SwfStudio) has been working on a Mac projector tool called FlashThing for some time now, so if that ever comes out then we can all be happy.
Posted by: Ryan Matsikas at May 8, 2004 10:14 AM
Hi Darron, that's true that you can use SWF in Mozilla across operating systems, but that's because both Mozilla and the Macromedia Flash Player have been ported to several platforms.
(The ActiveX technology, which IE/Win uses as an extension architecture, is indeed Windows-specific... there was a Mac SDK for awhile for IE/Mac, but in practical terms ActiveX Controls are for the Windows operating system. The Netscape Plugin architecture is platform-neutral, and any particular Netscape Plugin is platform-specific... the architecture specifies a way that you can add native-code extensions to any type of Netscape/Mozilla browser.)
If you want to run a XUL file, a SWF file, and possibly an HTML file on Linux, then just gather together your platform-neutral XUL/SWF/HTM and give them to someone who is running appropriate Linux-specific versions of Mozilla and the Macromedia Flash Player. Good...?
Regards,
John Dowdell
Macromedia Support
Posted by: John Dowdell at May 8, 2004 11:27 AM
Hi Darron,
I tested your example in mac OSX without any success. I think the problem is the communication between the player and the browser via fsCommand.
I checked the JavaScript Console and this error pops up:
Error: plugin.SetWindow is not a function
Source File:
javascript: function jsScriptObject(obj) { this.wrappedJSObject = obj; }
jsScriptObject.prototype = { evaluate : function(expression) { return new jsScriptObject(eval(expression)); } };
var plugin = document.embeds['flashapp'];
plugin.SetWindow(new jsScriptObject(window),102467287);
Line: 1
I think this is a very interesting subject and may be a good possibility for a multi-platform app.
Posted by: Nahuel at May 8, 2004 01:18 PM
Great idea, thanks for sharing.
Just give your window the attribute hidechrome="true" to hide all of the browser specific chrome.
Posted by: bokel at May 8, 2004 02:31 PM
Nahuel's got a point there... if you're also relying on plugin/browser intercommunication (ie, ActionScript and JavaScript passing messages back and forth) then there are indeed additional browser/platform dependencies.
FSCommand was available first for Netscape's original LiveConnect mechanism (on PowerMac+ and Win32+), then when an ActiveX version was available this could happen in IE3+/Win too. Mozilla changed intercommunication protocols awhile back, but I think it was Macromedia Flash Player 6 which added support for XPCom in Java-enabled Mac OS X and recent Windows (see technotes for details). I don't recall a Linux offering such intercommunication yet. One of the major problems in any use is in implementation: LiveConnect and ActiveX Scripting require two different scripting methods supported in the HTML page.
Summary: Yup, if your plugin tries to talk to your browser or vice-versa, then that could limit the portability of the project.
jd/mm
Posted by: John Dowdell at May 9, 2004 11:48 AM
Sorry, I was away for the weekend. Some comments of my own:
@Ryan - NorthCode announced FlashThing about a year ago, and it seems that still no progress has been made. Tired of waiting, I wanted to see if I could get something going on my own. :-)
@Nahuel - The Flash Player sends fscommands via the document.embeds object. The reason my "enable_fscommand" function exists is to eliminate that error. On my computer, I would get the same error until the fscommands were enabled (basically by just faking document.embeds, and having it point to the flash application). It might be possible that you have a slower computer and that enable_fscommand needs to be called after a longer period of time has elapsed. As I said, I didn't have a chance to test this on a Mac, but because Mozilla is cross platform, it should work in theory. Send me an email if you'd like to help me test to see if we can get a working solution. I'm optimistic that with a bit of tweaking we can get something going... :-)
@JD - The purpose of my post here is that I'm a little frustrated with Flash. I think it's an amazing tool for creating UI's quickly. However, it really lacks the power that it needs to function on the desktop. Screenweaver does an amazing job at giving Flash that extra little bit of functionality needed to create really useful desktop applications, the problem is that it's windows only.
By playing with Mozilla, I was attempting to find a solution that would bring that extra bit of power to Mac and Linux as well. The plugin only talks to the browser because the browser can then, in turn, do things that the plugin can't since the plugin isn't powerful enough to do things on it's own.
What I would LOVE to see are two versions of the Flash Player - one for "flashlets" running on the web, and another for "flashapps" running on the desktop. Think Java applet/application here.
I know Central is supposed to go down that road a bit, but I'm really not a big Central fan. I don't like not being able to create my *own* application with Flash.
Do you see what I'm getting at? Because Netscape is platform nuetral, and because XPCOM enables interaction with the OS, it seems natural to marry the .swf plugin with XPCOM to create cross platform *desktop* applications, running on top of Mozilla (but transparent to the end user).
Relying on Netscape communication is no different than someone needing the JVM installed or the .NET framework installed. As long as you have the minimum requirements, your code will work on all systems. I don't see how your summary above really applies, sorry. If there's a problem in the communication, then it needs to be reported to the Mozilla team (or the Flash Player team) so their "cross platform" statement stays accurate.
Posted by: darron at May 9, 2004 02:39 PM
By the way, I tested this on Linux (Suse 9 with Mozilla 1.6) and, as expected, it WORKED! Heck yeah.
I was able to, with Flash running on top of Mozilla, open a local file, make changes to it, and then save it back to disk. A "desktop" Flash app in Linux, is this the first one? :-)
Anyone care to donate a Mac to me so I can test on that? ;-) Are there any other Mac OSX users out there that got it to run sucessfully, or have the Mac users all been experiencing the same error Nahuel saw? Again, I'm optimistic that it will work on a Mac, since it already should, in theory.
Posted by: darron at May 9, 2004 03:23 PM
What a great idea!
Seems to work like a charm... :-)
..Just great!
Posted by: Roland Schaer at May 10, 2004 03:53 AM
If this works with the Mac without hacks, I think it is HUGE. If it can be made to work with the Mac by any means, it is still VERY BIG. Does Mozilla.org know about this? Could they help?
Posted by: Ron Phillips at May 10, 2004 07:28 AM
You're right, the Macromedia Flash Player is designed for efficient delivery of websized loads, rather than for creating desktop applications. If the goal were the latter, there's a whole raft of, not only features, but architectural decisions which would have been made differently.
(btw, was I reading correctly that you implemented some LiveConnect work in Mozilla for Linux!?)
jd/mm
Posted by: John Dowdell at May 10, 2004 02:27 PM
Darron,
I couldn't agree with you more. Very little, well actually nothing has been done by MM regarding flash desktop applications. To be fair it's a new arena, but it won't require a big architectural change. Just give us a projector with a simple API, even a small subset of the central commands would suffice for starters.
We wrote a OS X desktop projector environment for our software Gush (www.2entwine.com) that used cocoa and webkit but now we're being forced to look at a mozilla-based solution due to problems with safari. Getting a stripped down version of mozilla running on OS X unfortunately isn't as easy as it should be :)
Posted by: Wes Carr at May 11, 2004 10:12 PM
hey, nice work darron... i don't know how i missed this one. i have a project in the bin which is strikingly similar that i basically stopped working on because commercial opportunites started springing up. I'll clean it up and make a post... next time i find u online, i know what the topic will be :)
great work!!
Posted by: eokyere at May 14, 2004 04:29 PM
A bit OT, sorry, but:
Just had a quick glance at the sourceforge page, and found this really funny:
"Additionally, SharpFlash was built with SharpFlash, demonstrating the power of the program."
Looks like a "feasible impossibility", how can you build something with itself..? :)
Posted by: eyezberg at May 16, 2004 10:11 AM
Hallo: i had the same problem with mozila 1.7
{
Error: plugin.SetWindow is not a function
Source File:
javascript: function jsScriptObject(obj) { this.wrappedJSObject = obj; }
jsScriptObject.prototype = { evaluate : function(expression) { return new jsScriptObject(eval(expression)); } };
var plugin = document.embeds['flashapp'];
plugin.SetWindow(new jsScriptObject(window),102467287);
Line: 1
}
Is there any possibility to solve it?
thx.
Posted by: Laco at May 20, 2004 05:12 AM
This is a great post. I had the same thought about a month ago to solve a problem for a framework I am building for Ruby. It allows Ruby to drive a flash movie and create AS 2.0 components without modifying the Flash movie itself. From Ruby you can do stuff like this:
require 'alph'
include Alph
manager = Manager.new
manager.register(:main) do |flash|
rubyButton = flash.new_component("mx.controls.Button", :label=>"Close!!!", :_x=>143, :_y=>89, :icon=>"icon_16x16_warning")
rubyButton.setSize(120,22)
rubyButton.click {
exit
}
end
require 'runner'
AlphRunner.start(manager)
That is Ruby code, that when run, displays a projector with my alph.swf file in it and acts as a language bridge between Ruby and Flash. I can create any Flash object from Ruby, and call any method. I can also receive events (see rubyButton.click do...end above). Anyway, this framework is going to be released soon, and will be presented at OSCON in July. The main problem is that the projector is just the standard one from MM. It would be nice to have a cross-platform projector that could be controlled by my framework because then you could create multiple 'windows' and position them, filling in the content inside (components, etc).
I would really enjoy collaborating on this. Do you want to start a Flash-XUL project on sourceforge? Perhaps Gozerian...from GhostBusters (a la XUL) ;-)
Posted by: Rich Kilmer at June 3, 2004 09:15 AM
interesting post, .... cant you embed flash in director and then run on a mac - for file access etc?
Posted by: david doull at June 18, 2004 01:36 PM
interesting.... i'm not a big fan of fscommands tho...
looks like if u can make binaries controls mozzila can understand, u can have this swf talk to ure controls for functionality you would like to hide in binaries or use the advantages binary/compiled code offer...
very intersting...
nik
Posted by: Nik Khilnani at June 18, 2004 03:31 PM
coooooooooool idea
thanls for share your work !
Posted by: clement hussenot at June 18, 2004 04:24 PM
Hey...
theres a xul runtime environment being developed, maybe somewhat useful to realising this concept
http://www.mozilla.org/projects/xul/xre.html
Posted by: Jarrad at July 22, 2004 11:05 AM
I think it's great that JD (from MM) is posting here, but his comment, "Flash Player is designed for efficient delivery of websized loads, rather than for creating desktop applications" is way off the mark. Flash (and Macromedia) are spending big $$$ to steer flash towards a more rich client platform. I mean that is the whole reason for 2004 MX to exist!
I've been wanting to use Flash to write a non-web-delivered application that is cross-platform for a while. Hadn't seen Darron's post until today. I'll definitely be keeping an eye out for any progress in this area. Good work!
Posted by: Jim Cook at September 20, 2004 04:14 PM
I too was asked to deliver a flash desktop presentaion for cross-platform. One thing I wanted to do is use a fullscreen function. Of course not going to happen with any of the applications listed above.
We all know the mac projector generated in Flash MX publisher is limited. Some of you ask why MM hasn't contributed to this cause. If I were to guess it would be that they feel that desktop apps is a job for Macromedia Director not Flash.
The truth why 3rd party developers do not invest their time in creating Mac versions of their compilers is that no one will invest the dollars for 3-4% of users with those users being limited to mostly creative agencies, audio engineers and students. I still feel it is worth it but I still need to ask. Can this project be done without the need to "piggieback" on Firefox and flash plugin to access core funtions of the OS? If not then it will never be the same as the PC counter parts and I ask can a project like this get get a return of investment of all your time? Or is it another open source project that will take a long time to finish due to individuals developing it on their free time? Tough one. But I have 3 PC's and 1 mac OS X Panther machine. If any of you need to test anything let me know. peace.
Posted by: elstinko at March 1, 2005 07:59 PM
Hi, I realise there isn't any activity on this at the moment, but I've just found this! I am developing a multi-platform compilable app that will be using Flash as it's main document type.
The main problem I have hit now is displaying these flash movies much as the problems above.
It would be great to hear any updates or thoughts on this, I would be interested in finding out how to incorperate GRE into my app, and ultimatally wrap it up in my own custom widget...
Posted by: ZuluWarrior at May 4, 2005 04:51 AM
Any updates on getting this to work on Firefox/OS X? I would love to be able to use this with an app I'd like to develop.
Posted by: matriculated at August 29, 2005 12:47 PM
I looked into the problem and source and it's doable to have a standalong projector using the netscape plugin on any platform. Since different platform has a different plugin, the projector program will need to be compiled for the different platforms also. (but from the same source). I was thinking of creating a flash based login for Linux similar to my current FrontMotion Login for Windows.
Posted by: Eric at September 17, 2005 12:18 AM
How is this going? I've been researching swf wrapper techiques for a while and have learned various techniques, but found as you have the best may be to wrap the Flash mozilla plug-in. That has licensing issues,but if I've interpreted licensing correctly, I don't think Macromedia would have a problem if the wrapper doesn't actually include the plug-in. So our requirements for an application would say "Requires that Firefox with Flash"
So, all we all want is an extensible cross platform desktop framework or set to wrap the standard Macromedia distributed Flash plug-in.
What I've decided is:
1) Use a swf2exe like Multidmedia Zinc to create a Win and Mac wrapper to load ANY swf. Eventually, they're supposed to release a version which publishs a Linux wrapper. I don't like have to wrap EVERY swf, so I've taken care of that by using an xml config file to specify which swfs to load
2) Buy a copy of Director for PC and a copy of Directore for Mac and create two swf wrappers, essentially the same technique as with Zinc. The difference is that each copy of Director will cost me 1200 bucks.
3) Contribute to the open-source project, Screenweaver and hope that will speed progress
4) GRE... and that's when I ended up here. I think I like this option best, but I've got to make more progress to decide for sure. How are you doing?
Posted by: Michael at November 21, 2005 09:11 PM
Awesome! This is the first resource I came across on how to embed swf flash in an xul application-- great for a kiosk, standalone projector, etc. I also enjoyed this line:
> The "Windows-only" aspect of it really bothered me, and I didn't want to continue to devote time to something that left out a large portion of the Flash population.
I like that development is naturally heading more standards/inclusive, and you stated it well right there. Three cheers!
Posted by: bronius at February 23, 2006 11:08 AM
Have you seen ASUnit? They have a XUL based Flash UI that is cross-platform compatible and even runs in eclipse :-)
Posted by: Caleb at March 22, 2006 06:48 PM