It's a Sign!

A couple days ago, I was browsing the Stick Figures in Peril group on flickr. It's a great group, totally worth a look. There I spotted this lovely British "Elderly People" warning sign. There was much speculation, what the lady on the sign was doing, maybe she was picking the mans pockets? For me it was obvious: they were dancing Letkajenkka, they just were a bit lonely.

So, I got them a couple friends and here they go:
(Click on the image to see the animated version)

Original picture from theCallowQueen from flickr. Licensed under Creative Commons Attribution-Noncommercial-Sharealike 2.0 license.

I traced the figures and created a couple new poses in Inkscape, after which I composited the image using GIMP.

Openmind 2007

Looks like I'm a bit late blogging this, as Openmind was held already on 2. - 3. October and now it's end of the month.

Nevertheless, Openmind is an conference focusing on open source. It was held in Tampere, Finland. There were several interesting speakers. Most memorable for me were probably the talks on OpenMoko and Neo1973. Oh boy, I want one of those gadgets myself...

So, what I was doing there? Well, this conference is organized by COSS, the very same organisation that organizes Kesäkoodi. So, there was a session where every Finnish summer coder – or at least those who had reached the conference venue in time – presented their projects.

I gave my presentation on Inkscape there, too. It was a short talk, some ten minutes or so, covering what is Inkscape, what are filter effects and what improvements I made to Inkscape during summer. If you'd like to see this presentation, the slides are available at the Mindtrek website. They are also planning to post recordings of the sessions there. I don't know if these Kesäkoodi presentations were filmed, at least the camera was there. I'm now quite sure, if I want to see myself on video, though...

Large improvements in filtering framework

I've been rather busy lately, and it's been cutting into time, I can spend developing Inkscape. Nevertheless, I've now committed a big filtering framework update. The filtering code can now access the bounding box as defined in SVG standard, not only the visual bounding box, that's used in many parts of display code. Also, the filtering code can now handle many more different situations involving rotated and/or skewed objects. This update has been needed for some time already, but I thought it might be too big to fit in this summer.

The most visible problem this fixes is a crash when trying to use blending modes on a rotated or skewed object. It transformed the object to a coordinate system, where the object's x- and y-axis were paraller to bitmap axis. Then it tried to blend that with the background image, that was still in display coordinates. The blending code then tried to fit these in same buffer, which became really huge and couldn't be reserved.

This also fixes several less visible bugs. If an object was rotated/skewed and the resolution for filter was specified, the specified resolution was discarded and automatic resolution used instead. Filter area calculations are more correct, because they can be made in correct coordinate system.

This also opens new parts, that should be improved. The filters, for which it makes a difference, what direction the object's axis point, should be tagged as such. Cutting the filter result to filter effects area should be implemented. A better sampling than nearest neighbour should be implemented for pixel block transformations. Probably several others too.

This modification isn't all good, though. In some cases with rotation or skewing, the image quality is visibly detoriated. Also there's a rather interesting effect with low-resolution filters. In the following images, first has a slight blur applied with 16x16 resolution. Second is same image, but with a slight rotation applied. This is caused by the fact, that for scaling there exists a bicubic scaler, but for more complex transformations there is only nearest neighbour filtering available. Actually, this behaviour doesn't even violate SVG standard, but it definitely is rather unintuitive. (also, if you look closely, you can notice an off-by-one bug with the resolution)

Assembly 2007

Once again, the Finnish demoparty Assembly has ended. Well, actually it ended already on Sunday, but anyhow.

It was really great to meet all those friends who were there, watch the demos and other compos, check out the booths etc. This year I had again created a intro to be shown at Assembly. Well, it was shown. The entries in 4k compo were of really good quality, and my style is obviously a bit odd, so it ended up on 10th place out of 11 entries shown. For those, who are interested, grab the intro: tsunami.tar.bz2, complete with source code. It's under GPL, naturally ;)

On the not-so-nice sides of this years Assembly, I caught some flu there and because of it, I wasn't feeling too well for the time of the party.

And for the rest... Well, I don't know what I should write. And of course, it may not always be the best to write everything for the whole world to see.

It's alive!

I just a while ago committed feMerge support in Inkscape SVN. Now, we have all the parts required to render the example image in SVG specification!

Several other example files are now rendered correctly, too. This filter primitive doesn't bring anything really new, because everything it does, can be done with feComposite, feBlend or even with the simple alpha compositing used in SVG outside filters. It's just an easy way for filter creators to composite several images into one.

Also, check out the Filter effects page in Inkscape wiki. There's a nice listing, which filters work and which don't. There really is a good amount of filters that already work.

Lighting examples

Well, I got around and made an example image with lighting. This utilizes gaussian blur, diffuse and specular lighting and feCompose modes arithmetic and in.

New Inkscape developers

I haven't gotten much code done for Inkscape lately. Small bugfixes here and there, but nothing big really. Most of my time has gone into developing a 4k intro for Assembly 2007 and helping out new Inkscape developers.

Yes, indeed. There are two new guys, who have jumped into developing filters for Inkscape. Thanks to them, we have three new filter primitives implemented in Inkscape: feConvolveMatrix, feDiffuseLighting and feSpecularLighting. Once again, these open up new possibilities for artists using Inkscape.

Now I propably should toss in some examples using these new filters. I don't have any, so I'll toss in the Filter effects example from SVG specification. It shows one possible use for these lighting effects. Actually, Inkscape is almost able to render that one correctly, only feMerge filter primitive is missing.

For feConvolveMatrix then, it can be used for effects like blur, edge detection, embossing, sharpening.

That 4k intro even has a connection to Inkscape: the engine is able to render bezier shapes and I've used Inkscape for designing those shapes. More of that later, so I don't spoil it for anyone ;)

Introducing feComposite

I decided to postpone turbulence filter for a while and go ahead with my original plans. So now, Inkscape has a new filter primitive, feComposite.

Composite in this case refers to Porter-Duff compositing operators, introduced in a paper "Compositing Digital Images" by T. Porter and T. Duff. These compositing operators can provide quite interesting results, though I'm yet to come up with a example where they could be used. Well, one thing useful could be cutting holes to objects, for which using boolean operations would not be feasible.

Reading that paper, so I could implement the filter, a strange feeling of perspective crossed my mind. Even though computer graphics is a rather new area of study, this paper has been written before I was even born. Only a year before, but still.

Creating this filter I came upon several parts of filtering code, that should be simpler to use. Especially handling temporary images requires too much boilerplate code. So next, I will look into improving handling temporary images.

Timidity power usage

A while ago, I tried profiling my computer power usage with PowerTOP. This showed, that Timidity was waking up 100 times per second. As I wasn't using midi for anything at the time, this seemed rather odd. Tracing running Timidity with strace, I noticed that the alsa sequencer interface was polling something.

Digging Timidity cvs tree, I noticed this polling was introduced, when stream tracing support was added to alsa interface. Stream tracing allows for displaying Timidity state in real time, so it can't just sleep until something happens. For normal usage as software midi interface, this is useless. Before this tracing support was added, Timidity did sleep until some event occurred.

So, this patch makes Timidity alsa sequencer interface poll at 100 Hz only, when stream tracing is used. Otherwise, it just sleeps until something happens. This doesn't seem to break anything and considering that it doesn't actually add new functionality, I believe it's safe to use.

The patch: nopoll.diff

I can has turbulence?

feTurbulence is a rather problematic filter to create. It would be certainly be useful for same reasons gaussian blur is useful: it's hard to achieve the same result with only vector shapes.

Well, the first problem is, that the description of this effect in the SVG standard is rather vague. It mostly consists of C-source, which implements this effect. Well, this code doesn't even compile with gcc. It has only a couple useful comments and they describe the simple RNG used for this effect. What I have, is code that presumably calculates the colour value for single pixel, if I manage to figure out, what exactly I should pass it as parameters.

The second problem is a long-time PIA. The filters rendering code receives a bounding box, which is almost but not quite what SVG standard defines as bounding box. Well, for non-rotated and non-skewed object coordinate systems at least. With this effect, one fills the whole filter effects area, which is usually specified relative to the bounding box. (default is to expand the bounding box by 10% to every direction) As feTurbulence fills this whole area, it's really important to have correct size for this area. Otherwise, this will result in inconsistency between different SVG viewers, potentially also between different Inkscape releases.

In the Mood

This weekend, I was at the PuistoBlues blues festival in Järvenpää, Finland. Not only as a visitor, either. A couple days before the main concert I was building the tents - one for the VIP persons, a couple for serving beer, the backstage etc. On the day after the main concert, I was then tearing those same tents down.

I'm not too big a fan of blues, though I've become more interested in it in the last couple of months, as I've been learning to play harmonica. For me, this has more to it than only blues. It's a big festival in my former hometown and the artists there are world-class. The best thing still is the atmosphere on days before and after the main concert, as most people there creating the festival are volunteers. They (or actually, we) are not building the festival for money but because of the ambitions towards blues or the festival.

Well, the bands there were good. Once again, I didn't like the biggest stars Johnny Winter and Keb Mo so much as those playing before them. Or maybe at that point I was already too tired to pay attention. Well, anyhow. A Finnish band J. Karjalainen Lännen-Jukka was really good, though some people nearby me were pondering, if it was blues at all. It wasn't anything anything too polished, but more like simple country style music. J. Karjalainen himself was playing banjo along with a violinist, guitarist and a mandolin player.

Jody Williams & Billy Boy Arnold played more regular blues. Before their show, what caught my attention was, that Billy Boy Arnold is a harmonica player. This was a big point for me, I wanted to hear and see how he played. And were they ever good! I liked especially their pianist, in red shirt and hammering away on a grand piano.

There was also a local group, the Super Blues Band, who had also played at the first PuistoBlues thirty years ago. Quite a feat, I have to say.

A quick update

Just a quick update this time, I haven't finished anything especially noteworthy lately, so I haven't blogged about them either. The bugs mentioned in my last post have been ironed out, as well as some others. I did make the blur slider such that it didn't create a full new filter every time it was invoked, but as I committed that change, it became apparent that Nicholas Bishop had been working on the same code for his GSoC project on filter effects UI. So, that my code was obsoleted in a matter of hours. Re-implementing it, so it can work with this new blur & blending mode UI, could be useful though.

My plans now are to implement new filter primitives. My original plan states feMerge and feComposite, but as turbulence and lighting effects were considered really useful, when I discussed this on Inkscape Jabber channel, I might implement them first.

Dropshadows with filters

Today I had my last exam this semester. Well, sort of. Actually this semester ended already in the beginning of May and this was just a chance to retake exams. Anyhow, the test was on Design of Algorithms. In itself, this is really an interesting area of computing science and I'm planning to do my master's studies in the algorithms sub-programme. For this course then, I'm getting a bit frustrated with it. In the first exam I had no idea, how to do operations on binomial heaps, and since that would have given one third of the points, it didn't go well. For this exam then, a quarter of points would have come from remembering an approximate solution to traveling salesman problem, giving at most double the length of ideal solution in polynomial time, and how to prove it won't give longer answers than that. I didn't remember that solution, so this time it won't go too well either. (and I didn't quite have the time to build the solution from scratch... ;)

In coding Inkscape, I had a lot more success. Today I committed code that allows for using SourceAlpha and BackgroundAlpha as input images in filter effects rendering. These are the image of filtered object and its background, but with all colour information stripped out.

I really have no idea, what I could do with BackgroundAlpha, but SourceAlpha at least gives me nice dropshadows. This star here, for example (can you tell, I like using stars as example material?-). It is just a green star, with a filter that takes the BackgroundAlpha image, offsets it by 10 units in both x and y directions, blurs the result and finally blends that image with the original green star.

There are still some oddities with that offset filter. I tried this first without blur, and it resulted in white gaps in the whole shape, which was a known bug for the offset filter. Not good. For my amazement, everything went just fine, when I added the blur. No banding, no spurious messages about trying to reserve almost 16 EiB of memory. I really need to take a better look into what's wrong with that feOffset renderer.

Simple filter, big problems

feOffset is a rather simple filter primitive: it just moves the given image a given distance. If we don't go into sub-pixel rendering accuracy, the framework in Inkscape makes it possible to implement a renderer for this filter by merely updating image location info. So far so good.

What's the problem then? Well, even for simple images rendering this filter requires us to actually render another part of the picture: suppose we wanted to render the rectangle from (x0, y0) to (x1, y1). When we have a feOffset filter with distance (dx, dy), we actually need to render area (x0+dx, y0+dy) (x1+dx, y1+dy) and then apply the filter.

This can be accomplished, rendering blur needs similar functionality and it works quite OK. The bigger problem is, when feOffset is used on background image. In this case, at the time when background is drawn, it's not known, that some filter later on will try to use background outside this area. All the ways to remedy this I can think of, are rather complex. Still, this whole background access functionality is a rather complex subject. It will likely require some kind of re-entrant rendering, that can go back to rendering background when this kind of situation arises.

Also I've spent a way too much time trying to fix background access functionality, that wasn't broken in the first place. Obviously it's dangerous to modify one's own old code, because it's easy to assume remembering its workings. If that was someone else's code, I would have first taken a good look at how it works.

Making use of Inkscape

I'm spending plenty of time developing a drawing program, so I decided to draw something for change.

This drawing is based on a photograph, but only to get the anatomy correct. I started with ordinary A4 copier parer and pencil for the first sketches. For the outline drawing I used heavier A4 drawing paper, on which I first drew the subject with pencil and then inked with a brush pen and 0.20 mm pen.

Scanning was a bit of a problem, as my scanner seems to be breaking down. The scanned image was full of horizontal colored lines. Luckily all those lines were bright colours, so I was able to open the image up in GIMP, take a CMYK separation of the image and discard all but the K (black) channel.

From this on, I used Inkscape for the work. Tracing the inked line drawing gave me a nice vector representation. This was actually surprisingly simple, as in GIMP - which I have used for drawing, too - separating these lines from background would have taken many steps and still wouldn't have yielded as well-defined lines.

The new paint bucket tool is a godsend for colouring drawings. Filling areas of drawing with the basic colour is only a mouse click away. With default settings, the filled area is not quite correct, as white lines show up between the area and bounding line, but this is easily remedied by increasing the value of "Grow / shrink by".

For adding shadows, I got to use the new featured I had created myself (Yay!). For this drawing I had though, that the lighting is quite warm, like summer day light, so I picked a cold color for the shadows - dark blue to be exact. Simple alpha blending with this kind of colour leaves a bit to be desired, so I used feBlend with multiply mode. Each shadow shape has an opacity of 20% instead of the whole shadow layer having that opacity - this way stacking shadows on top of each other will create darker shadows.

The few brighter spots are also created by using feBlend, this time with screen as mode. I feel that bright spots are really easy to overdo, so they are quite subtle here - some maybe even too subtle.

Well, for the bubbles and background I used plenty of gradients - probably not hard to tell ;) The bubbles are all clones, so it was easy to edit their appearance, when they didn't seem quite correct.

The pier planks are all unique, the tops are created by duplicating one plank, though. Also, lots of guidelines were used here. This is a place, where I wished that the 3D tool wasn't still in planning, but an usable tool.

And yes, the result:

I was also thinking, that I might create some kind of tutorial about this, too. I did take screenshots and save several copies of the image along the way. This might be a bit too large subject for a tutorial, though.

And yes, I haven't totally forgotten coding. As pointed out in inkscape-devel mailing list my last changes to codebase broke the about screen... Mixing these C and C++ objects is tricky sometimes. This time problem was that a C++ object inside a C object wasn't getting initialized. If both objects were of C++ style, this initialization would have happened automatically.

Filtering the image background

It's beginning to seem, that my planned timetable for this summer project is way off. Luckily it seems, that I've overestimated the time I need instead of underestimating. So what does this mean exactly? Well, now it's been a full two weeks since I started the project but I had planned five weeks for doing what I've done up to this point. Also, I've planned two weeks for creating renderer support for BackgroundImage and BackgroundAlpha input images. Well, it seems that support for BackgroundImage exists already, it just needed these changes I committed today to enable it.

Well then, about this commit today: support for in-parameter in filter primitives. This means, that now we aren't limited to filtering only the image itself and building linear filter chains. The background image can be taken into account when filtering and building complex filters should be possible too.

There are lots of possibilities that this enables. Next picture shows one of them: frost glass effect. Here I've got an ordinary gaussian blur, but instead of applying it to the object itself, I apply it to the background image. The object itself disappears, all we see is that a square area of the image is blurred.

This support for different input images isn't without problems, though. One really annoying one is with blend filter: it sometimes doesn't use the specified blending mode, but seems to apply normal alpha blending. I don't know yet, if this is a bug in input images support, in blend filter or even in some seemingly unrelated part of code.

To help debugging this, I think I'll reorder my timetable and implement feOffset filter next. This will allow me to use some example images W3C has published and compare renderings from Inkscape to them.

Refactoring work

First planned work for this summer was to refactor filter renderer initialization. The old initialization code was a monolithic piece of code copy-pasted in three places in codebase. (nr-arena-shape.cpp, nr-arena-group.cpp and nr-arena-image.cpp) While not bad in itself, each new filter primitive implemented would have grown that code by a dozen lines or so. It would soon have grown to an unmaintainable size.

So, the new approach: SPFilter has a single method sp_filter_build_renderer, which will initialize given renderer object (NR::Filter) to a correct state. Calling this method is all that needs to be done in those three nr-arena-* classes to set the correct filter renderer state.

The inside workings of sp_filter_build_renderer are as follows: each filter primitive (SPFilterPrimitive subclasses) has a build_renderer virtual function that will add the correct NR::FilterPrimitive object in the filter renderer. This function in turn calls sp_filter_primitive_renderer_common, which will do the part of initialization, which is common for all filter primitives.

Also, I looked into how modification messages are handles in document level filter objects. This actually took more time than the actual refactoring work, as it required me quite a few recompiles and plenty debugging to figure out, how these messages are propagated. Well, now these messages are propagated from SPGaussianBlur and SPFeBlend. This means that updating the underlying xml nodes for these two filters will cause the image on screen be updated. This will allow for further improvements, for example making the blur slider update the existing filter instead of creating a new filter every time the slider is changed.

Now that I've done this refactoring work, I will look into making in-parameter for filter primitives work. There is now the single method sp_filter_primitive_renderer_common where I should make this change to, so it will be somewhat simpler than it would have been before this refactoring work.

Also, I probably should write documentation, telling basically the same things about this refactored initialization routine, I've told in this blog post, just more detailed.

Rather odd blending bugs

Not long after committing the feBlend renderer, ScislaC found some rather odd bugs in it. Well, I'll show you the pictures:

What it looked like:

What it should have looked like:

So, what was up with this? Well, due to roundoff errors, under some circumstances feBlend renderer would output colours with more than 100% of some RGB components. The image compositing code didn't take such colours nicely...

Well, now this should be fixed in SVN.

On other topic, today I was looking for prices of new memory modules, so I could make this computer run smoother. It seems that increasingly often 512 MB just won't do it and the hard disk trashing begins. Now then, SO-DIMM memory is a bit on the expensive side, but for a bit less than 150 € / GB it would likely be worth the money...

Setup phaze and feBlend

Today I finally wrapped up some modifications I had lying on my hard disk and committed feBlend renderer into Inkscape codebase. The codebase surely changes fast, as testing those changes with latest code required a full recompile. (and that does take quite a while)

That feBlend renderer isn't much of use yet, as it needs some features, I'm planning to develop over the summer - for example, changing the input image. Blending the filtered image with itself isn't too useful a feature...

Recompiling Inkscape will hopefully take significantly less time in future though, as I installed distcc, which allows me to offload part of the compilation effort onto my other computer. It's quite nice a tool, as it works nice even though that other computer doesn't have all the libraries Inkscape requires, different versions of some and even a different processor architecture.

Inkscape and I

A bit over a year ago, I noticed I could apply for Google's Summer of Code program. Browsing their list of projects, for which I could apply, I bumped upon Inkscape website. Graphics programming has been one of my interests for a long time and Inkscape was a program that I was using already now and then. After discussion with Inkscape developers, I ended up submitting my proposal on creating infrastructure for SVG filter effects support.

So, the proposal got accepted and during the summer I worked on developing filter effects support in Inkscape. Soon it was fall and I returned to my studies at the university. I did commit some code during the winter, too, but not really much. When the spring came and I noticed ads promoting Kesäkoodi - a Finnish program similar to Google's Summer of Code - in our student's room, I thought it would be great to continue what I did last summer.

So here I am, beginning another summer filled with reading code, planning, writing documentation - and of course, writing code, too.