Putting it all together part 1

So far I have created three different components for this project; an algorithmic tree generator, a program for pulling colours out of a webcam feed, and finally an algorithm for downloading RSS feed weather data. At some point these individual parts need to be fused together, and that point is now.

So what do all these things do? Well firstly the top left image shows the webcam feed, bellow is a compressed version of this, and finally, below that is the most common colours found in the image. On the right-hand side is the digital forest. The colours from the webcam feed are mapped directly to the colour of the trees. The features of the trees themselves at this stage is still fixed.

Bristol weather feed

Moscow weather feed

At the top of the app, the current weather data is being printed, and this can be specified in real time by switching out the RSS feed on the fly. Currently, the temperature is mapped to the 'ground' colour of the forest. If for example, the temperature is below 0 then the ground will turn white.


My main remaining challenge at this point is extracting features from the image to specify the generation of the various trees. A secondary aim is to further make use of the incoming weather data by mapping it to other features within the forest.

Getting Weather Data

After talking about the project in last weeks session, it was decided that it would be great if the virtual forest could react in real time to some data inputs. In the real world, external factors from the environment such as wind have an instantly visible effect on trees, other factors including temperature & humidity take time to realise.


In theory, the installation could include several sensors for measuring environmental parameters using physical hardware sensors, however, it would also be interesting too position the virtual forest in different real-world environments. i.e. what would this forest look like right now in California or Syberia?

RSS

As defined by Digital trends [1], RSS stands for 'really simple syndication'. At their heart, they are just simple text files with basic updated information — news pieces, articles, that sort of thing. In this instance, an RSS feed can be used to get up to date and succinct information of the current weather forecast. 

Example output from a BBC weather RSS feed.

So the next question was how do I get an RSS data feed into a C++ program. A quick search of GitHub revealed a number of open source libraries that would have been suitable, however, I really wanted something simple and not full of external dependencies.

Some more hacking

Making use of what I already knew I was able to hack together a fairly simple application in C++ that could get the current weather report. I shall now talk through the code line-by-line.


The first line of code simply makes a call to system, followed by the curl command, which is used to pull down the content of the RSS feed and pipe it into a plain text file.

The program then waits before trying to open the file.

Make sure the file is actually open before trying to parse it.

Next up we simply go over the file line-by-line 

We then use the following code to check to see if we have got to the weather description part.

Once there we make a call to the stringBetween function to pull out various bits of information, the std::stoi converts a string into an integer.

Finally, we can print out the information we extracted.

The entire program looks like this:

I can now think about putting the various parts of the project together!


[1] https://www.digitaltrends.com/computing/what-is-an-rss-feed/

Camera Input

Another part of my project that I have started to work on is the leaf scanner. In that, the installation will scan in leaves and extract some features from the image to be used as input parameters into the tree generator.

Back at the start of the semester I went out and gathered some 'literal' training data in the form of physical leaves. I then laminated these up thinking it would help preserve them...

Laminated leaves!

Unfortunately, I did not dry them out before laminating them, so they have now started to decompose. I may have to resort to using prints instead.

4 weeks later!

To scan the image in I simply made use of an openFrameworks tutorial that connects to and uses a webcam. From this, I can get access to the raw image data. The first thing I wanted to do was to analyse the image and find the most common colours.

To achieve this the image was first captured as a still. The image was then clustered into squares of 4x4 pixels and the average RGB values calculated. This initial clustering reduced the image by a 1/16 from 921, 600 data points (for a resolution of 640 x 480) into 57,600 points, which is far more manageable! From this, each of these values could then be stored in a matrix. Finally, this data was then piped into a K-means clustering algorithm. The centres of each centroid gave them most commonly used average colour values. Later in the development stage, this data will be propagated into the virtual trees to give them their colour.

Application showing input image left, compressed (1/16) image right. The blobs at the bottom are the average colour values from the K-means process.

White box with the camera placed at the top for capturing image data. Highlighters provide some colourfull test data!