Friday, December 21, 2012

Emscripten News: LLVM 3.2, etc.

LLVM 3.2 was just released today, and as with every LLVM release emscripten is switching to it. (We only support a single LLVM version at a time; if you don't want to upgrade to LLVM 3.2 just yet, you can use older revisions of emscripten.)

LLVM 3.2 brings as usual a large amount of general improvements and bugfixes. There isn't anything in particular that will be noticeable with emscripten, except for a change in how LLVM does linking. It now requires an explicit list of symbols to keep alive - this was quite puzzling to me at first but respindola explained it, and this is a very nice change for LLVM to make, linking is more consistent there now. We do have all the necessary symbol information in emscripten, but were not passing it to LLVM, now emscripten has been modified to do so.

The result is that in some cases more unneeded code can be removed, resulting in smaller generated code, which is great (for example ammo.js is 2% smaller). However, if you do not explicitly keep a function alive (either by using EMSCRIPTEN_KEEPALIVE or __used__ in the C/C++, or adding it to EXPORTED_FUNCTIONS), then LLVM may remove it.

Another improvement is landing together with this into master, unrelated to LLVM 3.2, is better linking of .a archives. We now only use the object files that are actually required, and will not link in others. This can also reduce the size of generate code, but again, if you are not careful, needed functions may be removed, in particular because the link order of archives matters (libA.a libB.a will only link in parts of libA that were required by things before it on the commandline, not things in libB).

Finally, another change you might notice if you use emscripten is that it now has better support for systems with both Python 2 and 3 installed at the same time. ack wrote a big patch to make our usage of python much cleaner to enable that. One significant consequence is that we now look for python2 in the python script shebangs. So if you run ./emcc and do not have python2 in your path, you will get an error. Solutions are to either run python emcc or add a symlink to python2 from python.

5 comments:

  1. I am regarded by most as insane. However I think I manage cool stuff occasionally.

    Backstory: Qt's QML was raster based. As of Qt5 it is OpenGL ES 2.0 based. OpenGL is somewhat of a mystery to me, though I know the basics of how it works.

    Here is my consumdrum. whereas I could use a HTML5 canvas and a QPainter that spat out HTML5 JS canvas commands, I can no longer do that for Qt5/QML applications as they all use OpenGL now. To I'm trying to pull the same rabbit out of my ass except in GL land.

    QMLServerApp(GL) - > ??? -> WebGL -> Browser(ANGLE) -> Client (GL)
    I am aware that it can potentially be reduced:
    QMLServerApp(GL) -> Client (GL)
    However it would require a custom EXE and therefore native deployment.

    The QMLServerApp(GL) - > ??? -> WebGL -> Browser(ANGLE) -> Client (GL) solution is the best even though features might be lost in each layer.

    I'm looking for the inverse of the Browser portion (???). You probably have heard of OnLive.com - where they do streaming gaming. Imagine then, that they don't send you a h.264 stream but only a compressed stream of GL commands. The bandwidth utilization would be lower.

    Anyway, your blog seems to indicate that you would be the definitive person to talk to about this.

    Do you have any suggestions on how I am to proceed?


    Many thanks.

    ReplyDelete
  2. There is a GTK backend that streams draw commands to an HTML canvas, you might want to look at that for a similar approach and solution. In general, sure, you could stream GL commands, but they are not very network efficient (there are many each frame). Possible though with enough work.

    ReplyDelete
  3. Yes, I did look into Broadway (that is what that is called) and I did a test o it for Qt which was largely successful (I called the Qt version Vaudeville) but I need to complete it. Supposedly I was going to be able to at work. But this has not happened yet. However WebGL is a different beast, though it has similarities.

    ReplyDelete
  4. Hi, I'm having a problem with the make.py script. I've build the emscripten, but I'm not sure it is connected to the Broadway.
    Anyway, that is what I'm getting:


    C:\Program Files (x86)\Microsoft Visual Studio11.0>python C:\Users\DELL\Broadway-master\Decoder\make.py
    Traceback (most recent call last):
    File "C:\Users\DELL\Broadway-master\Decoder\make.py", line 9, in import tools.shared as emscripten File "C:\Users\DELL\emscriptenmaster\tools\shared.p, line 461, in EMSCRIPTEN_TEMP_DIR =tempfile.mkdtemp(prefix='emscripten_temp_',diconfiguration.TEMP_DIR) File"C:\Python27\lib\tempfile.py", line 329, inmkdtemp _os.mkdir(file, 0700)WindowsError:[Error 123] The filename, directory name, or volume label syntax is incorrect: 'C:\temp\\emscripten_temp_ttqvgs'

    ReplyDelete
  5. No one will see your comment here - you should file a github issue.

    ReplyDelete