Youtube 1080p / ClickToFlash

Youtube announced they are adding 1080p videos besides the existing HD (720p). In order to enhance playing performance of H264 videos embedded in a Flash object (or simply disable these), we can use ClickToFlash (forwards the decoding to the QuickTime plugin).

Logic Express 9 on G4

Installing Logic Express 9 on a G4 (MDD 1,25 Ghz) finally didn’t require any change to the installer. Moreover performance seems good, as it could run the demo song (by disabling some tracks). It uses 9,7 Gb space (4,7 Gb of EXS samples, 900 Mb for Ultrabeat loops, 3,5 Gb of GarageBand instruments, 500 Mb of AppleLoops, and 720 Mb for the application – after the 9.0.1 update).

Blog record : more than 5000 visits / Adsense

The blog just exceeded 5000 visits in July (that is exactly 5240). During its first month launch last September it still accounted for 1700 visits, and grew very fast (4400 visits in December, and in 4000-4800 range since). It also gained pagerank 3 very fast. The main Cjed Home site (that opened early 2005) also gets 5000 visits/month (with a max at 8000 visits) and a pagerank 3 level (reached a pagerank 4 peak two years ago).

However since March 2006 the whole gains from Google Adsense are only 50$ (that is for 3 years and a half of targeted ads, and close to 3000 news. Hopefully the purpose was to review Google’s technologies). Google bots are also late to index (despite use of urlrewriting and news titles injection in the html page title) : it sometimes can take up to a week to index new articles, while this WordPress blog gets its new content indexed in less than 3 hours !

Google AppEngine limitations / workarounds

Here are some limitations to consider when we plan using AppEngine. They are most likely due to clouds constraints and consistent performance concern. We can read a compatibility list of various J2EE frameworks and supported Java APIs.

1000 files limit
Each application is limited to 1000 files. Then when using Cappuccino we have to use Press tool (with flatten option – to be tested with 0.71 version as it was broken in 0.70 beta) or remove the .j files (keep only the .sj). For CP2JavaWS, images resources from unused components were also removed (load time should be however better using Press).
As the concept of physical machine is gone with clouds computing, we cannot write in files, and we can only read files in WEB-INF (or accessibles through the classloader).

1000 results limit / limited offset for requests
For performance concerns, requests results are limited to 1000 records. Moreover the offset cannot be higher (requests return 0 records when the offset reaches the 1000th position). Then we cannot browse a thousands of lines table with limit(offset, count) if not using a filter/condition.
We could still add a criteria on the index to fetch records in multiple parts (for each thousand step). However this would only work if we are sorting by the index (wouldn’t work if sorting by another column). It is the same if using two requests (one for retrievinig the indexes and another using selection where index in (index range) limit count), as the first request result will be limited. Same problem if we use a temporary sort table (the result from the select request used to retrieve the data to insert is limited), moreover that solution requires to compute again the temporary table if the sort criteria (or condition criterias) vary.
One solution would be to define a special key (see Python section Queries on Keys, _key_), in memory managed and without read limit (same section for Java/JDO doesn’t however include these informations). It would require to be able to modify dynamically this key (as it depends from the sort column), or to define for each sort column an additional column (setted alongwith each insert), composed with the sort column value and index value. We could then add to the request a _key_>previous limit value criteria (value of the composite column fot the last element retrieved during the last 1000 elements fetch). Adding a criteria on the primary key wouldn’t work if using another sort column, as the criteria applies before sorting (that problem is worked around if the criteria uses the composite column that corresponds to the sort column).
We can however assume that search criterias have to be refined/tighten if more than 1000 results are expected.

Mapping limitation / caches synchronization
Hibernate isn’t supported as it instancies statically a UUID generator, using inetAddress class (is among the restricted/unsupported APIs by AppEngine, as well as other machine related instructions : threads, etc.) Then a modified version of an Hibernate class is provided in CP2JavaWS (uses code from JUG framework instead).
The demo of CP2JavaWS uses an HSQLDB in-memory database to easier the example install (no databse server to create), and table and initial data are created at launch time (from a context listener). The values of the third column (age) are generated randomly, in order to allow testing of sort feature. Then these values can change depending the timeframe we access the application (as these values are in memory, they are different from an application instance to another). That isn’t a problem however for that demo (no persistance required).
We could configure an url to a database server, however it would have to be hosted elsewhere (if not datastore). And we cannot also use a local file to persist the database (only read allowed).

The main concern is memory synchronization between cloud nodes (application instances), as mapping solutions use two objects caches. The first level cache (one per persistanceManager/MappingSession – typically per user session) allows to compare a working object copy with the corresponding original (fieldLocking), and second level cache (one per persistanceManagerFactory/SessionFactory) allows to compare orignal object from persistanceManagers with current corresponding objects in the central cache (that is necessary for optimistic locking. Direct access – back door – to the database without passing through the persistanceManagerFactory – that is generally retrieved from JNDI – is forbidden, in order to keep integrity).

The GAE datastore is based on the DataNucleus mapping framework (implements JDO and JPA). By default the level 2 cache is off in DataNucleus, however by activating it (through settings) we can choose among various implementations : EHCache, Oracle Coherence, memcached, etc. In that list only Oracle Coherence and memcached allow work in a distributed environment (second level cache replication) – also the case for more recent versions of EHCache. Thanks to the plugin architecture in DataNucleus we can develop extensions to use another cache framework that manages distributed mode : JBoss cache, OSCache, Terracotta, etc.
The Google datastore uses a proprietary implementation of JCache specification (JSR107) to allow a distributed mode, and manages replication automatically. DataNucleus provides a manual API to manage datastores replication (for example the JDOReplicationManager to synchronize PersistanceManagerFactory). The AppEngine SDK also provides Memcache APIs, to manage (manually) replication of custom objects if needed.

Replacing DataNucleus with Hibernate (can use distributed caches like JBossCache, OSCache, Coherence and more recent versions of EHCache) in a GAE application would require to have hooks into the replication process (synchronize the HibernateSessionFactory). The distributed caches configuration (static) would however require to know exactly the nodes hosts, and GAE doesn’t provide such information.

Finally we are tied with the Google datastore, with the following restrictions (do not come from Datanucleus limitations) : no aggregate requests, no polymorphic requests, limited filters, limited joins, limited many-to-many relations support, etc.
As BigTable isn’t relational, JDO looks interesting as it isn’t restricted to relational datastores (contrary to JPA). DataNucleus manages various datastore types, and an extension (plugin) to BigTable had to be developed by Google (notably to manage access through JPA interface). Despite the GAE datastore being at a higher level of abstraction than BigTable, some limitations seem directly tied with that implementation choice, whose goal is to provide consistent response time whatever the request (thus the above limitations).

No support for SOAP webservices
This shouldn’t be a problem as recent RDA solutions (GWT and Cappuccino/CP2JavaWS) use JSON (enhanced with proprietary fields).

Inter-applications communication
Applications have to use URL Fetch APIs from AppEngine SDK to communicate.

Tools and WTP integration
The Eclipse plugin allows to create a web project (with AppEngine webapp configuration file, and required jars – have to be added manually to the buildpath), but doesn’t provide integration with WTP (and no mean to stop the AppEngine server once stared – have to use WTP stop button). We can in fact deploy an AppEngine project from a WTP project, by renaming WebContent to war, and by adding the appengine-web.xml file to WEB-INF folder of the WTP project. However we then haven’t support for classes enhancement (required for the datastore JDO and JPA implementations). We could probably add this automatic task by adding the appropriate builder in the .project file.
The enhancement step required for the mapping adds to the previous constraint of client code generation if using GWT. Some JDO solutions like LIDO still allowed to remove the enhancement step if needed (leading however to some performance hit as expected).

Cappuccino performance on iPhone/fat binary

Cappuccino performance on iPhone isn’t satisfying for now, due to the device’s processing power and limited memory (that are also main reasons for the absence of a full Flash version – besides strategic matters and historic poor peformance of Flash player on mac compared with the Windows version). From the Cappuccino team leader, it appears that WebKit team may have discovered a bug in iPhone’s Safari that leads to an additional parsing of each js file, thus requiring twice the time (moreover that step takes half the whole time of a CP application load). Besides that large improvement area (that could be provided soon through the iPhone 0S3.0 update), the Press tool could filter on a method basis instead of on a whole class basis. Indeed some classes such as CPArray include a hundred of methods, that aren’t likely to be used all in a given application. However this could be problematic with dynamic calls, where used methods aren’t known before execution.

Then it is planned to allow deploying Cappuccino applications targeted for various environments (desktop, mobile). A new archive format (that reminds OSX universal binary concept) will be provided (no compilation however there). The Atlas editor could then produce archives optimized for all devices.

Use of Cappuccino for portable applications is a straight choice however, as if performances are a concern, migrating to iPhone SDK would be easy (same frameworks and concepts).

Myst for iPhone released : 727 Mb

Myst for iPhone is finally available at the AppStore. Moreover it is reasonably priced (5,99$). Its size is huge (727 Mb – that is more than the original game, whose resolution was higher but with a lower bits depth) and the install process requires 1,5 Gb of space left on the device (this additional space is retrieved once the install completes). Here are the features list :

• All the original Ages & gameplay
• High quality images (better than original Myst)
• Full music & sound effects
• Original movies & animations
• Auto-save (when quitting or phone call)
• “Bookmark” system to save & restore progress
• Swipe to turn
• Auto-zoom in certain close-up areas
• Quick access to hint guide

It works with iPhone 3G, the original iPhone, and iPod touch.

UPDATE : about saturated and distorted music (sound effects are ok however) it seems the DS version also experienced that problem (moreover with worse graphics) :

(the sound effects and music are crackly and distorted…)

EastWest PLAY 1.2 update

A new PLAY update, 1.2, is available at soundsonline. It notably brings an enhanced streaming engine. However performances don’t seem really better on G4.

Geekbench : MacPro Nehalem 8 cores / G4 1,25Ghz

First tests of new Nehalem MacPro through Geekbench are very encouraging : the 8 cores/2,92 Ghz (high end model) is globally 23 times faster than a single 1,25 Ghz G4 (18 times faster on integers, 32 times faster on floating point calculations, 10 times faster for memory access, and offers a 14 times larger bandwidth).

However, there has been globally few progress since G5s (were unveiled close to 6 years ago).

TVOut/Screensplitr : iPhone video on external device

ArsTechnica provides an article about iPhone video mirroring solutions. TVOut software provides the best framerate but doesn’t manage all applications (no games and no OpenGL support). Screensplitr framerate is far more lower but it is compatible with all applications and games.
These two softwares require an AV video wire (see the iPhone AV/USB from USBfever, that also allows battery charge). However a new feature in Screensplitr allows to get the iPhone video from a browser (Safari, Firefox), then wireless (through Wifi), and even to take control of the device !

FileAid free/DiskAid : mac/iPhone transfer through USB

FileAid application for iPhone (text and media visualizer, including PDF and Office formats) is available for free at the AppStore until March, 17. We can also download on the same editor site, DiskAid, that works on mac and allows files transfers to/from iPhone through USB. To remember, the combination of Discover and DiskAid didn’t allow to access files (DiskAid doesn’t have access to the WebDAV document root folder used by Discover for transfers). A Wifi connection was then required.

Apple : security Guide for MacOSX

Apple published a security guide for MacOSX. It is very comprehensive (260 pages), very technical, and targeted for advanced administrators, for sensible context (administrations, defense, etc.) :
This document is intended for use by security professionals in sensitive environments. Implementing the techniques and setting found in this document will impact system functionality and may not be appropriate for every user or environment. 

The document introductory lists the new security features added by Leopard :

Better Trojan horse protection. Mac OS X Leopard marks files that are downloaded to help prevent users from running malicious downloaded applications.

Stronger runtime security. New technologies such as library randomization and sandboxing help prevent attacks that hijack or modify the software on your system.

Easier network security. After you’ve activated the new Mac OS X Leopard application firewall, it configures itself so you get the benefits of firewall protection without needing to understand the details of network ports and protocols.

Improved secure connectivity. Virtual private network (VPN) support has been enhanced to connect to more of the most popular VPN servers without additional software.

Meaningful security alerts. When users receive security alerts and questions too frequently, they may fall into reflexive mode when the system asks a security-related question, clicking OK without thought. Mac OS X Leopard minimizes the number of security alerts that you see, so when you do see one, it gets your attention.

Tutorial : install OSX on PC through EFI-X key

The site materielBoys.fr provides a step by step guide about installing OSX on a regular PC through the EFI-X USB key (150$). The only problem encountered was the video quality while playing movies with the DVD Player, however it doesn’t seem to affect all graphics cards.