Fennec 1.0 gource

Fennec 1.0 has been released for the N900 devices.  It is fun to look back to see how far everything has come.  One interesting way to view the development progress is a visualization tool called gource.  Gource produced this graph for the frontend changes that make up Fennec 1.0 (best viewed in HD).

There are lots of nobs and switches to gource.  The command to produce this video is:

gource -s 0.001 –output-framerate 25 –highlight-all-users –stop-at-end –disable-progress –auto-skip-seconds 0.1 –file-idle-time 10 –max-files 1500 –multi-sampling -1280×720 –stop-at-end   –output-ppm-stream – | ffmpeg -y -b 30000K -r 60 -f image2pipe -vcodec ppm -i – -vcodec libx264 -vpre hq -crf 28 -threads 0 fennec.mp4

A group of N900's

what do you call a group of N900′s?

A bunch of N900

Labyrinth Game in the browser

A few months ago, I mentioned the game Labyrinth and how I wanted this to work in the browser. Syd Lawrence took up the cause and put together a cool implementation.

Play it here. You must be running Firefox 3.6 RC or better.

This uses the new orientation events added to Firefox 3.6.

Syd also put together a demo of what orientation looks like for those that do not have Firefox 3.6:

This web feature is going to be standardized in the W3C in one of two working groups.  The general idea is to separate out acceleration (values in g) and orientation (values in degrees for azimuth, roll, pitch).  More on that next year when we figure out where the work will be done!

ncat ascii art

config.status: creating config.h
(  )   /\   _                 (
\ |  (  \ ( \.(               )                      _____
\  \ \  `  `   ) \             (  ___                 / _   \
(_`    \+   . x  ( .\            \/   \____-----------/ (o)   \_
- .-               \+  ;          (  O                           \____
)        \_____________  `              \  /
(__                +- .( -'.- <. - _  VVVVVVV VV V\                \/
(_____            ._._: <_ - <- _  (--_AAAAAAA__A_/                |
.    /./.+-  . .- /  +--  - .     \______________//_              \_______
(__ ' /x  / x _/ (                                  \___'          \     /
, x / ( '  . / .  /                                      |           \   /
/  /  _/ /    +                                      /              \/
'  (__/                                             /                  \
NMAP IS A POWERFUL TOOL -- USE CAREFULLY AND RESPONSIBLY

Fast Flash in Fennec

For the last few days, we have been making Flash fast in Fennec.

Our situation in Fennec is that we have a hidden browser element that contains the flash object/embed element. When a paint needs to happen, we draw whatever the plugin wants to draw into a canvas or sent of canvas elements. These canvas elements are what the user sees on the screen — they are part of Fennec’s tile manager. Now these draws to the tile manager consistent of a 16bpp to 24bpp conversion (I am told that the flash is optimized for 16bpp), then a copy to a gfxXLibSurface, then a final blit to the screen. This final bit also contains a 24bpp->16bpp conversion because the screen is 16bpp. To make matters much worse, many plugins intersect multiple canvases in the tile manager which causes this drawing path to happen multiple times per video frame. The end result is that we were getting no more than 4-5 fps.

We took some incremental steps to improvement performance, but we were no where close to double digit fps.

At this point, we decided to just draw directly to the screen avoiding the tile manager completely. This allowed us to render without any conversions and only one copy — the plugin could write directly to X11 shared memory.

In making this decision, we would lose a bunch of information that Gecko provides such as where the plugin should be position relative to other elements, and how it should be clipped.  The solution we came up with was to let Fennec tell each of the object and embed elements where they should be drawn.  The frame painting code would honor the position and clip that the front end set.  In this way, we could have plugins do the right thing during pans and when content is below the Fennec sidebars/urlbar.

You can check out how Fennec positions elements here:

http://mxr.mozilla.org/mobile-browser/source/chrome/content/browser.js#2937

The end result in this work is that we get over 25fps when Sorenson encoded videos.  Over the next weeks and months, I hope to see more video content honoring Fennec’s user agent and provide optimized content.  (YouTube and other sites do not recognized Fennec’s UA, yet).

This fast path is only implemented on Maemo, but it can be implemented on any platform assuming the plugin has support to draw directly into a memory buffer.

You also can check out the details in bug 528551.

Geolocation support in add-ons

Today, I checked-in a set of changes that will allow mozilla addons the ability to acquire the location information. It is pretty simple to do:

geolocation = Cc["@mozilla.org/geolocation;1"]
.getService(Ci.nsIDOMGeoGeolocation);
geolocation.watchPosition(successCallback);


In this trivial example, the object |geolocation| is fully compatible with the W3C’s geolocation object that exists under the navigator object in web content. |successCallback| will be called when Firefox acquires your position.

The prompting of the user is left up to your application. All addons hosted on addons.mozilla.org must follow the guidelines and prompt the user.

If you have questions about what is permissible, please contact me.

Location Privacy

A month ago, I was at the MetaPlaces conferences on a panel on Privacy. Dev, the moderator, asked a really interesting question: “what are you most scared about”. It is a very interesting question. Sitting next to me was the head lawyer of the CDT, someone who has giving me tons of grief about the w3c’s approach of geolocation on the web. He joke that my biggest fear should be not listening to his advise on this. However, I am a lot more scared of something else.

The previous day at MetaPlaces, I heard a lot about mobile advertising and the targeting operators can provide. What was most scary for me was the amount of information operators have and their use of this information to place you into a very detailed market segmentation… all of this without your expressed permission…

One company was able to take a two week data drop from an undisclosed operator and tell you the sort of lifestyle, socioeconomic status, age range, and other demographics of the phone owner. The data drop merely consisted of longitudes and latitudes of where the phone was at given times. Following a single phone you are able to known where the person lives, what kind of coffee he drinks, what area the person works, what stores he shops at, what their work hours are like, are they hitting clubs at night or are going home, and do they spend time at the library. And the user is aware that this sort of tracking is happening!

This is wrong. Operators should always be up front about this. The location data is yours. Where you take your phone, like who you call, is personal information.

Recently, a group of privacy advocates are calling on Congress to address some of these concerns. Some of the more interesting requests are:

* Sensitive information should not be collected or used for behavioral tracking or targeting.

* Individuals should be protected even if the information collected about them in behavioral tracking cannot be linked to their names, addresses, or other traditional “personally identifiable information,” as long as they can be distinguished as a particular computer user based on their profile.

* Individuals should have the right to confirm whether a data controller has their personal or behavioral
data, request such data, and delete it.

For more information about this effort, please check out the CDD press release.

how i made the 192 pushlog ugly

in other words, how i screwed up a hg push. I wanted to push changes from mozilla-central to mozilla-1.9.2. The standard way is to:

=============================================================
how to apply a m-c committed patch to a release branch
=============================================================
hg qimport http://hg.mozilla.org/mozilla-central/raw-rev/
hg qpush
hg qref -e # add approval information
hg qfin .
hg push

What I did was basically the same thing, with a few shortcuts (insert your favorite story about how cutting corners can cause harm).

Basically I started out with a bug report. It has the information about what it fixed, who reviewed it, who approved it, etc. The first thing to do is verify that it all checks out and is a reasonable thing to do. In the bug, there usually a comment with the hg changeset information. Here was my first shortcut; I copied the url to the html version of the changeset from the bug, and run the qimport command. `hg qimport` has no problem with importing that patch (which should probably generate an error, but anywayz). What this did was imported a big html file — not what I wanted at all.

I then jumped over the `hg qref -e` command. This would have allowed me to see the changeset’s original comment. I would have been able to see that the qimport terribly failed. Also, this is a good idea to do so that you can add the appropriate a=foopy approval flag(s).

The last shortcut I took was I didn’t do a sanity check by running `hg out -p` which would have shown me what I was pushing. This is always a good idea to see what you are pushing. Sure reliance on the tools is fine and dandy, but actually seeing what hg thinks it is going to push doesn’t hurt and only takes a few seconds.

3 shortcuts resulted in a skidmark on the 191 push log:

http://hg.mozilla.org/releases/mozilla-1.9.2/rev/739c7a69d412
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/1898753917d5

Sure there are worse things (e.g. patch bomb, pushing without testing, breaking trees), but I thought I would write this up both for clarification of how I messed up as well as a warning.

CoreLocation in 10.6

I took a really quick look at the built-in location services on Mac 10.6.  I wanted to determine what sort of accuracy the feature had out-of-the-box, and what the API looked like.

The code is pretty simple.  Just something like:

CLLocationManager* locationManager = [[CLLocationManager alloc] init];
locationManager.delegate                = self;
locationManager.desiredAccuracy = kCLLocationAccuracyBest;
[locationManager startUpdatingLocation];

The delagate is just two functions:

- (void)locationManager: (CLLocationManager *)manager
didUpdateToLocation: (CLLocation *)newLocation
fromLocation: (CLLocation *)oldLocation
{
}

- (void)locationManager: (CLLocationManager *)manager
didFailWithError: (NSError *)error
{
}

If you put this in your code, you magically get a prompt that looks like this.  “Untitled” is the name of my application.

Location Request for "Untitled" application

Clicking on the Help icon, you get the following:

Help screenshot

The network requests goes to mac-services.apple.com.  The good news is that it is over a secure network channel.  The bad news is that the content is also hidden from inspection.  (eg.  since this is close source, Apple could be passing anything.  The thing that worries me is that it might be passing the application name which would give unfair Apple insight into what applications are popular).

The accuracy is pretty good — it places me at the building I am working from.  Now where is the support for Geolocation in Safari? 

Using CoreWLAN on MacOS 10.5

When Apple’s OS X version 10.6 shipped, they broke compatibility will all 3rd party geolocation applications.  This wasn’t entirely unexpected as we all were using a private framework which wasn’t documented.  So, Firefox 3.5 and Gears, and probably Skyhook, are busted right now on OS X 10.6.

Apple has provided, with limited documentation, a new public framework called CoreWLAN.  It looks pretty easy to use.  To do a scan of the WIFI access points, you can simply do:

#import <Cocoa/Cocoa.h>
#import <CoreWLAN/CoreWLAN.h>

NSError *err = nil;
NSDictionary *params = nil;

NSArray* scan = [NSMutableArray arrayWithArray:[[CWInterface interface] scanForNetworksWithParameters:params error:&err]];

This is all fine and dandy.  However, Firefox uses 10.5, and so using the above code will not compile using the old SDK.  Unfortunately, Apple didn’t provide any C apis – only exposing this objective-C API.  So, what I had to do was some objective-c magic.  Check out the reference (http://developer.apple.com/mac/library/documentation/Cocoa/Reference/ObjCRuntimeRef/Reference/reference.html)

Basically, we needed to be able to dynamically load the CoreWLAN library, that is simple enough:

void *corewlan_library = dlopen(“/System/Library/Frameworks/CoreWLAN.framework/CoreWLAN”,  RTLD_LOCAL);

Now that this library has been loaded, we want to simulate the objective-c call “[CWInterface interface]“.

// get the class
Class CWI_class = objc_getClass(“CWInterface”);
// register the selector name
SEL interfaceSel = sel_registerName(“interface”);
// make the call
id interface = objc_msgSend(CWI_class, interfaceSel);

This mess above is the same as:

id interface = [CWInterface interface];

As you probably can see, it isn’t that hard to do this, just it is syntactically nasty.  Here is the source to the little application I was testing with:

#include <mach-o/dyld.h>
#include <dlfcn.h>
#include <unistd.h>

#include <objc/objc.h>
#include <objc/objc-runtime.h>

Class CWI_class = objc_getClass(“CWInterface”);
printf(“cwi class %x\n”, CWI_class);

SEL interfaceSel = sel_registerName(“interface”);
printf(“sel: %x\n”, interfaceSel);

SEL scanSel = sel_registerName(“scanForNetworksWithParameters:error:”);
printf(“scanSel: %x\n”, interfaceSel);

id interface = objc_msgSend(CWI_class, interfaceSel);
printf(“interface: %x\n”, interface);

id scanResult = objc_msgSend(interface, scanSel, 0, 0);
printf(“scanResult: %x\n”, scanResult);

Hope this helps.  Please let me know of an easier way, if one exists.