Category Archives: Uncategorized

Do I upgrade to Fusion Energy?

I have been thinking a lot about where my energy comes from.

20051029_Belchatow_power_stationI used to be quite happy with fossil fuels from GWT Co: They were so cheap, reliable and freely available.

But I found out that not only are they going to run out quite soon… they are also doing bad things to my environment!

 

1280px-Alternative_EnergiesSo I have switched energy companies to use one of those cool new green tariffs from ngGreen. Now a large proportion of my energy comes from renewable sources, such as Wind and Solar (and possible nuclear fission!).

 

 

Alcator_C-Mod_Tokamak_InteriorRecently, I heard that there is this new renewable energy source, based on nuclear fusion. Apparently the Tokamak facility managed to achieve 30 seconds of sustained fusion. In the future there is no doubt that we will all be relying on this cool new technology to provide all our energy needs.

But how do I get hold of this clean sustainable energy; when should I switch and how can I plan for that?

Well the truth is that workable fusion technology is not here yet. It will be great when it comes but right now it is not a viable option. So I keep making the best of the renewable resources out there and try to limit my usage of fossil fuels.

AngularJS – Services

angular-services-bookAngularJS brings many useful features to JavaScript application development, such as unobtrusive data-binding and the ability to add functionality to HTML through directives but one of the often overlooked killer tools is the dependency injection framework. By decoupling the construction of code components from the components themselves developers are able to focus on the actual business functionality of their components.

At the core of AngularJS dependency injection is the concept of a service. Due to some unfortunate naming conventions and the ability to define services in a number of different ways, this concept is often confused and underused with Angular application developers pushing too much code into their controllers.  Services are a key element of structuring your application effectively – both for testability and maintainability.

In his new book, Angular Services, Jim Lavin guides the new and intermediate AngularJS developer through all the service related concepts and describes numerous good practices for defining, testing and using services in your Angular apps.

I like how the programming principles are laid out in general as well as showing how this is implemented in the Angular environment. Much of the book could apply to almost any dependency injection framework. I was also pleased to see so much focus on testing and testability. This is a key software engineering topic that has traditionally been overlooked in many JavaScript development projects.  Hopefully the reader will be inspired to work more testing into their own projects after finishing this book.

As with any set of guidance the reader should consider each piece of advice carefully as there is no such thing as a one-size-fits-all approach to software engineering. There is plenty in this book to think about and work into your own projects. Even the most advanced programmers are likely to find something useful in these pages.

AngularJS Docs Performance

I noticed at a recent Meetup in New York – http://youtu.be/-Z-NuSklwWU?t=39m38s – that the performance of the AngularJS docs app was questioned and quite rightly so. Loading up the docs app was a complete dog!

docs-before

As you can see from this Chrome timeline the page is taking over 4 secs (on a fairly powerful laptop with fast broadband connection) to complete loading and rendering.

The mystery of the blocking script

What is going on here? Well the actual loading of the various files is not too bad (though it could be better). Most of the files are loaded within 1500ms. But then there is this mysterious block of script evaluation happening from 1500ms to around 4000ms. What is worse this script is blocking the whole browser. So the user experience is terrible. On a phone you might have to wait ten or more seconds, looking at a pretty blank page, before you can start browsing the docs.

In the video Brad and Misko suggest that the problem is with a Google Code Prettify plugin that we use to syntax highlight our inline code examples. It turns out that while this is a big old file the real problem is elsewhere. Notice how the memory during this blocking script goes up and up with a number of garbage collection cycles throughout? This is due to a memory intensive process.

Houston… we have a problem with the Lunr module

A quick look at a profile of the site loading show us that the main offenders are found in the lunr.js file.

profile-before

Lunr is the full text search engine that we run inside our app to provide you with the excellent instant search to all the Angular docs (API, guides, tutorials, etc). Unfortunately, we are creating the search index in the browser at the application bootstrap, which is blocking the rest of the application from running and stopping the page from rendering.

I looked at various different options to fix this. Initially I thought, “Let’s just generate this index on the server and download it”. Sadly the generated index is numerous Mb in size so this is not acceptable (especially if you are on your 3G connection)! I also looked around to see if there were alternative search engines we could use. I didn’t find anything that looked like it would be more performant that Lunr.

Getting async with Web Workers

So if this was no go and we must generate the index in the browser we needed a way to do it without blocking the rendering of the page.  The obvious answer is to do it in a Web Worker. These clever little friends are scripts that run on their own thread in the browser (so not blocking the main rendering thread), to which we can pass messages back and forth. So we simply create a new worker when the application bootstraps and leave it to generate the index.

Since the messages that can be passed between the main application and the worker must be serialized it was not possible to pass the entire index back to the application. So instead we keep the index in the worker and use messages to send queries to the index and messages to get the results back.

This all has to happen asynchronously but thats OK because AngularJS loves async and provides an implementation of the beautiful Q library, in the form $q service.

The code for the docsSearch service to set up and query the index looks like this:

You can see that since we are interacting with an async process outside of Angular we need to create our own deferred object for the index and also use $apply to trigger Angular’s digest when things happen.

The service itself is a function which takes a query string (q). It returns a promise to the results, which will be resolved when both the index has been generated and the query results have been built. Notice also that the query is not even sent to the Web Worker thread until the promise to the built index has been resolved.  This makes the whole thing very robust, since you don’t need to worry about making queries before the index is ready. They will just get queued up and your promises will be resolved when everything is ready.

In a controller we can simply use a .then handler to assign the results to a property on the scope:

The effect of this is that we now have completely removed the blocking JavaScript from the main thread, meaning that our application now loads and renders much faster.

docs-after

 

See now that the same block of script is there but now the solid darker rectangle is much smaller (only 0.5ms). This is the synchronous blocking bit of the script. The lighter paler block is the asynchronous children, which refers to our web worker. Interesting by moving this into a new thread it also runs faster as well, so a win all round!

But what about Internet Explorer?

There is always the question of browser support. Luckily most browsers do support Web Workers. The outstanding problems are the older Android browsers and of course Internet Explorer 8 and 9. So we have to put in place a fallback for these guys. I considered using a Web Worker shim but actually that would just add even more code for the browser to download and run. Instead I used the little understood service provider pattern to be able to dynamically offer a different implementation of the docsSearch service depending upon the browser capabilities.

The webWorkerSearchFactory, which we saw above, is provided if the browser supports Web Workers. Otherwise the localSearchFactory is provided instead.

This local search implementation is basically what we had in place in the non-performant version of the docs app but with a small user experience improvement. We still have to block the browser at some point to generate the index but rather than doing it immediately at application bootstrap we delay the work for 500ms, which gives the browser time to at least complete the initial rendering of the page. This gives the user something to look at while the browser locks up for a couple of seconds as it generates the index.

 

We could, of course, go further with this and chunk up the index building into smaller blocks that are spaced out to prevent a long period of blocking but since the number of browsers that this affects is growing smaller by the day it didn’t make sense to put more development time into this.

To cache or not to cache?

Some of you might be wondering why we are not then caching the generated index into the LocalStorage. This is certainly an option but it raises additional questions, such as:

How and when we should invalidate old cached indexes? We generally release a new version of AngularJS once a week; so it is likely that the cache will need to be invalidated weekly; also people may want to look at different versions of the docs, should we cache more than one version and how do we decide which ones to keep.

Also, LocalStorage has very variable and limited capacity. One has to put in place code to cope with storage running out. The indexes are rather large so this could happen quite often.

On mobile we probably don’t want to be caching multiple  large indexes anyway since storage is more of a premium in such devices.

Given the speed and user experience improvements achieved simply by moving the index generation to a new thread it was felt that this was a good enough solution for now.

Further Performance Improvements

In addition to this refactoring of the search index, we have also removed dead code, refactored other bits to lower the amount of data that must be transfered and minified much of the rest of the JavaScript. Our top man Jeff has also tweaked the server so we should be getting many more files zipped up when downloading now too.

If you have any other suggestions feel free to comment or even better put together a Pull Request to the repository with further improvements!

Fluency in Maths

I am a huge fan of the NRICH website – a resource “rich” site for anyone interested in learning or teaching maths. I regularly read through their monthly newsletters and have used it to support my kids’ maths development as well as a source of interesting mathematical bits and pieces for my own entertainment.

I am also aware that my kids’ school uses it occasionally for ideas for “Home Learning” topics (they don’t set “Home Work”).  For this I am very pleased because the kind of activities suggested on NRICH are so much more creative and fulfilling than the mindless of traditional maths exercises.

I have often debated what the most important and useful skills one should learn in maths; and although I have come round to the idea that knowing your multiplication facts (an exercise in memory akin to learning to recite poetry) can be useful in lowering the amount of work your upper brain needs to do when solving number related problems, I am more convinced than ever that a solid understanding of how numbers can be manipulated and perceived along with solid analysis and problem solving.

With that in mind I was really pleased to see a recent article on their site called Developing Number Fluency.  The sentence that caught my attention was “The first thing to say is that fluency is not only about number”.  So often people equate maths with number and number only.  That is such a shame.  It is like saying that literacy is only about spelling or music is only about scales. Maths can be creative and beautiful and fun and we should all be shedding our preconceptions and prejudices and looking deeper into what joys can be found in understanding patterns both in number and other areas.

Mocking Functions That Return Promises

I have been writing a funky little chrome extension for GitHub, which lets us view and update the labels on Pull Requests – a feature that would be easy for GitHub to implement but sorely lacking. You can check out this extension here: https://github.com/petebacondarwin/github-pr-helper

While I was writing unit tests, I wanted to test a directive that relied upon a service that returns a promise.  I wanted to mock out the service, itself, rather than get drawn into setting up mock $httpBackend responses.  Initially, I started writing out something like this:

This allowed me to test that the getCheckedLabels method had been called but then I wanted to get this spy to return some mock data via the promise.  I started thinking I was going to have to write a load of boilerplate code to store and then trigger the success handler in my test, which looked really ugly.

Then I thought I would be clever and create a generic mock promise object that I could use in all these situations with some kind of mockPromise.resolveWith('myMockData') API. As I was about to do this the obvious truth that this had already been done slapped me in the face.  In tests, I just want a promise that is either resolved or rejected already, so I just create one using $q:

Simple, eh?  This mocks out the getCheckedLabels method with a function that returns an already resolved promise.  The test is clean and simple and does what I want.