ninajansen.dk

I’ve just put out a new gem “cloud” that I’ve been working with on and off for the past few months. You can find it on github, it’s called ninajansen-cloud. I’ve also made a repository with examples which you can also find on github ninajansen-cloud_examples.

My motivation for this gem is that I love making wordclouds with wordle. However there are a bunch of things about wordle that I don’t love. It’s closed source, so you can’t take it apart and hack it. Also, it only runs in you browser as a java app. I wanted to be able to generate wordclouds as a background process. Eventually, I want to be able to make a cloud out of anything, like a table in a database. I want to be able to run cloud generating as a background process in my rails app. All this will not work with a closed source in-browser java project.

Now, I am not, nor is anyone else permitted to reverse engineer wordle, and I have certainly not done that. I don’t know what algortihm wordle code uses, it is certainly much faster than the one I came up with. Cloud is slow, can’t insert words inside other word, and can only generate pdf-files, which must be post-processed by some other code to make it into other formats. The only redeeming quality is that it is open source and I am really hoping that there are other people out there who would like to spend some time making cloud much better.

I use a 2D binpacking algorithm for cloud. To start with, I use pdf:writer’s built in pdf-functions to compute the size of a box that a word takes up. I add up the total area, and add some extra just in case, and find the smallest possible paper size that has this area and is avaible in pdf. Since pdf is scalable, you can always transform your clouds to some uniform size later, if you want. Once I have all the boxes, I take the largest and place it at the center of the canvas. I then add all the corners and centers of the sides of the box to an array. I look through this array of points to find the one that is closest to the “center” of the canvas. The center is computed by one of several distance functions, so it is possible to make clouds of different shapes. Once I have found the point closest to the center, I take the next box, and place it’s opposite point there (if the point is at the center of the righthand edge of the box, I place the center, lefthand edge of the second box at the point). Then I check if this placement is okay, i.e. if it doesn’t overlap with any other boxes. If it is okay, great, I have placed the second box and I can add it’s points to my “possible locations” array. If not, I try to place it at another point. I keep going until all boxes are placed. I clean up points regularly, i.e. I try to place the smallest possible wordbox at each point, and if it cannot be placed I erase the points from my “possible locations” array.

This algorithm gets slower and slower the more boxes (words) there are. The first 100 words are definitely faster to place than the next 100 words. 100 words takes about a minute. I’ve added speed to the algorithm by using ruby inline and written some of the functionality in C. Still, you wouldn’t want to generate very large clouds this way. But if you want to put a wordcloud on the front page of your website with treding topics, you could easily run cloud periodically in the background.

I hope that someone would like to use this gem, and contribute to it. You could add more color palettes or distance functions, incoorporate it into rails or just use it and tell me what you think.

Popularity: 75% [?]

So I wanted to experiment with Phusion Passenger on my home server. The server runs Ubuntu 8.10 and serves a bunch of small low-traffic sites through apache (including this blog). I saw that there was a package from brightbox for passenger on ubuntu and tried to install it. However, this package automatically disables the php5 apache module. I have no idea why, it is possible to run both php and passenger apps from the same apache. Anyway this clobbered all my wordpress blogs (I know, I should always read the messages carefully…) and I had to reinstall php5. If you want to install passenger on Ubuntu 8.10, use this approach.

Popularity: 72% [?]

So I am working on an application where I have an array where I need to the sum of the values of members of the array, if they fulfill certain conditions. It looks like this:

if !A and !B

array.each do {|a| if (some condition) sum += a }

if A and !B

array.each do {|a| if (some other condition) sum += a}

etc.

Now this can easily become much more complicated if there are more flags that can change what needs to be done inside the loop,  i.e. which members of the array should be added to the sum and which shouldn’t. For each possible permutation you spawn a whole bunch of code that essentially does the same thing. I solved this problem by using the eval function.

The eval functions takes a string and evaluates it as code. So for instance, eval(“1 == 1″) would return true. You can use this to make an array of strings with stuff that needs to be done inside that loop. As an example consider this:

conditions = Array.new

if params[:some_id] != 0 : conditions << “self.some_id == params[:some_id]” end

You can add more conditions, depending on parameters passed to the function and then join them together:

conditions_string = conditions.join(‘ && ‘)

Then you only need to make one loop:

array.each do {|a| if eval (conditions_string) sum += a }

You could, of course, join the conditions array with || or include the logical operator in the string itself.

Popularity: 67% [?]

In order to get a cache sweeper to actually do anything when it is triggered, it must be called from a controller, and that controller must be instantiated though a http request.

This is what I have learned from trying to expire fragment cache from background scripts in Ruby om Rails. It simply is not possible. If a controller is not instantiated when you hit the expire_fragment method
(or any other cache sweeper method) you get a nil reply.

I understand why this decision has been made: Fragments are in views. Only the controller should deal with views. The controller is instantiated when it is requested by the web-server. Otherwise you break the MVC paradigm. Except that this mental model does not take into account the fact that
every web-application (except the simplest ones) have stuff going on in the background. Long calculations are made by script so the user doesn’t have to wait while they are being finished. Stuff is done at a given time of day, like expiring old subscriptions. This is why we have BackgroundDRB and BJ. Background jobs need access to the fragment cache. If anyone who is really into rails could find a way to let background jobs do fragment cache things, I’d be very grateful.

For now, I have to periodically expire or reset all fragments because I know that background jobs periodically change the data used to generate them. Most inelegant and inefficient.

Popularity: 83% [?]

Today was the first day at railsconf, and I attended two tutorials: “Refactoring your rails application” and “CI for the rails guy/gal”.

First, I’d like to say that the tutorial format really doesn’t work well for large groups. Tutorials are best if the teacher/speaker can walk around the room and coach the attendants through a bunch of examples that illustrates the use of a tool or technique. When there are more than 100 people in the room the tutorial really becomes an extended talk. In that format it is better if the tutorial is an introduction to a tool or technique where the speaker(s) walk though the examples, but do not expect attendants to follow every move.

The “refactoring” tutorial had a lot of problems. The topic was interesting, and the speakers seemed to know a lot, but the problem was that attendants were expected to download an example application before starting, and then get that up and running on their own laptops. When 200 people try to do that at the same time over the same network it takes a long time. I was missing a gem (rcov) which took even longer to download, and while I was working on that, the speakers talked and I missed a lot. It was too bad. When I gave up on the application and focused on the speakers it got better. I never heard of the presenter patterns before, nor services, but both are something I want to look into further.

The tutorial on continuous integration was better, because the speaker presented the subject in a way so you could follow along and type in all commands, but you didn’t have to. The first half was very hands on: “this is how you set up your CI environment”, and the second half was pointers to more advanced topics. The material (a handout and a bunch of files which I copied from one of the DVD’s that the speaker had prepared and passed around) was excellent, and something that I am sure I will use when I start experimenting with this. I want to look into Selenium right away.

Tonight I will attend one of the BOF sessions, and then crash early, because tomorrow the real conference starts and I know from experience that there will be so much information to absorb that I’ll need to be well rested and sharp.

Popularity: 75% [?]

I use cache_fu for caching my rails application. I use a “fat models, skinny controllers strategy”. This means that my Author model has code like this:


def cached_books
get_cache(:author_books) do
Book.find_by_author_id(self.id)
end
end


This method caches all books belonging to an author. Cache fu does not cache belongs_to or has_(one|many) relationships, and that’s why we have this method. But this also means that if we add a book to an author, we must also expire the author records cache. This can all be done with cache_fu. The problem is that these relationships might get quite complicated and it would be nice to be able to write some specs that test that everything behaves as intended.

Chris Wanstrath, the author of cache_fu, suggests using mocha to stub everything about, but didn’t want to emulate the caching behavior and then test if the behavior is right. That is fine if you test the cache_fu plugin, but not fine if you need to test if your application uses the cache_fu plugin correctly. What I wanted was to turn on caching while running my specs and then query the cache to see if things are actually cached properly.

This first this I tried was turning on caching in the “test” environment. Turns out that it is not a good idea, as a lot of specs go into some kind of recursive error when you do that. Caching should be off by default in test. I needed a way to turn on the cache functions in a given spec.

Long story short: for unit testing you add the following function to your model:


class << self
def reenable_cache_for_test
class << self
alias_method :fetch_cache, :fetch_cache_without_disabled
alias_method :set_cache, :set_cache_without_disabled
alias_method :expire_cache, :expire_cache_without_disabled
end
end
end


You call this method in your spec like this:


it "should cache author" do
Author.reenable_cache_for_test

CACHE.flush_all
CACHE.get('Author:1').should == nil
author = Author.get_cache(1)
author.should != nil
CACHE.get('Author:1').should == author

CACHE.get('Author:1:books').should == nil
books = user.cached_books
CACHE.get('Author:1:books).should == books
end


Fragment caching is a bit different, in you controller spec you do this:


it "fragment should be cached on get" do
ActionController::Base.perform_caching = true
ActsAsCached.config.clear
config = YAML.load_file( File.dirname(__FILE__) + '/../../vendor/plugins/cache_fu/defaults/memcached.yml.default')
config['test'] = config['development'].merge('benchmarking' => false, 'disabled' => false)
ActsAsCached.config = config
ActsAsCached::FragmentCache.setup!

CACHE.flush_all

… and then do your cache checking.

Popularity: 100% [?]

So i’ve been spending hours tracking this one down: I wanted to have cache sweepers in rails and followed the advice from railscasts and railsevny. Those tutorials tell you to put sweepers in an app/sweepers directory and add that directory to your load path. DON’T DO THAT. That simply did not work for me (using rails 2.02). After banging my head against google for hours, I found out that sweeper simply has to go in app/models, MVC be dammed. If they are put there they get loaded, and you can add them as active record observers. If they are not put there rails won’t find them. I don’t know if this is a bug or a feature, but that is just the way it is.

Popularity: 94% [?]

So basically, the official plugin is old and cant really deal with restful routing. For some reason, when I used the url_for method in my views, the in place editor javascript function couldn’t interpret the (restful) route properly and sent a post request to ‘/controller/1/edit’ to an action named “1″ with an id set to “edit”, which obviously didn’t do anything useful. So instead I called the in place editor script in the following manner (I’m using haml btw.):


%span{:id => "name", :class => "in_place_editor_field"}= @user.name
= in_place_editor "name", {:url => '/user/edit/' + @user.id.to_s, :rows => 1, :size => 32 }


So basically I hard coded the route so it would fall back on standard routing. Yes I know that:

• This is ugly as sin
• I’m supposed to do a PUT request to a function named update, not a post request to a function named edit.

But it solved the problem.

Popularity: 94% [?]

I gave a talk yesterday at IHS, Sønderborg. I talked about astronomy/astrophysics and black holes but the conversation also took us into neuroscience, atheism and the Moon hoax (there is no such thing, we did go to the moon).

IHS is a “højskole”, which is a special Danish type of educational institution. People go there to further their education in subjects not covered in more academic types of schools. Typically you go for either a 6 month course or you can go for a week or two in the summer. IHS specializes in al types of sports, and the facilities wre great (I got a tour after I gave my talk). The students were really great and interested in what I had to say, and the conversation was lively.

My mindmap for the talk is here. It’s in Danish. I will make an English version if someone asks me to give a talk in English at one point or another.

It felt good to give a lecture again and see that I inspired the students, I haven’t done that much since I left science. I might be available for gigs in the future, so contact me

http://www.mindmeister.com/maps/show_public/6396492

Popularity: 77% [?]

These might be completely obvious to everyone, but I found them useful:

• If you want to migrate a specific database, you can do it by setting RAILS_ENV on the command line. Like this:
  rake db:migrate RAILS_ENV=production
• If you want to see all the rake tasks in your application with a description of what they do, execute:
  rake -D

Popularity: 64% [?]