Qui-Gon Jinn and the chance of being caught

January 26, 2011 – 6:00 pm

I love the movie reviews made by RedLetterMedia. They are irreverent and to the point, and they are not only a real pleasure to watch, but also an interesting insight in how movie critics is performed. My favorite review is from Star Wars – Episode 1 – the Phantom Menace. While I agree with the points presented in the review about the movie, there’s one thing that got my attention.

In the first scenes of The Phantom Menace, Jedi Master Qui-Gon Jinn and his apprentice Obi-Wan Kenobi need to escape from a ship and reach the surface of a planet, where the mission is to warn of an incoming invasion. Qui-Gon decides that the best strategy is to split and board different ships.

Plinkett, the rather “characteristic” reviewer in RedLetterMedia’s creation, points out this strategy as totally stupid: splitting would “increase the probability of getting caught by 100 %”. I disagree. Before I explain why, we need to lay out some assumptions.

The first assumption is that a man hidden on a ship has a, say, 10 % probability of being found by guards. The same probability is associated to a roll of a 10 faces die and obtain, say, the number one.

The second assumption we make is that the probability of getting caught stays the same regardless of the number of people boarding the ship. If only Qui-Gon boards the ship, or if he is together with Obi-Wan, the probability of being found is still 10 %. Apparently, this is not entirely reasonable: more people equals more noise, but more people means that if someone is discovered, the other one can kill the guard before he sounds an alarm. So, let’s assume that indeed the probability stays the same regardless of how many people are on the ship (within limits).

The third assumption is that ships don’t communicate. Suppose Qui-Gon and Obi-Wan board two different ships, and Qui-Gon gets caught. Under our assumption, this fact will not increase the probability of Obi-Wan getting caught. This is not true if, for example, Qui-Gon’s ship broadcasts a radio signal “we found an intruder, check your ships if you find others”, which would increase the probability of Obi-Wan getting caught due to increased security. Let’s assume that this does not happen, because in the movie the ships perform a deorbit procedure, which takes a small amount of time, not enough to coordinate such communication.

Given these assumptions, is Qui-Gon Jinn strategy of splitting a good one or a very bad one, as our reviewer says?

If they stay together, they have 10 % probability of getting caught, e.g. not being able to carry out their warning mission. If they split, there are four possible scenarios

  1. Qui-Gon is caught, Obi-Wan is caught. Mission fails.
  2. Qui-Gon is caught, Obi-Wan is NOT caught. Mission is successful.
  3. Qui-Gon is NOT caught, Obi-Wan is caught. Mission is successful.
  4. Qui-Gon is NOT caught, Obi-Wan is NOT caught. Mission is successful.

We need to find out what is the probability for each of these cases.

As we already said, the chance that a man on a ship is caught is assumed to be 10 %. Due to the laws of probability, the chance of two independent probability events A and B jointly occurring (such as throwing two dice) is the product of the probabilities for each event

P(A and B) = P(A) P(B)

meaning that the chance of both Qui-Gon and Obi-Wan are caught (case 1) is

P(Qui-Gon caught and Obi-Wan caught) = P(Qui-Gon caught) P(Obi-Wan caught) =

= (10/100) * (10/100) = (1/100) = 1 %

which is, basically, the probability of failure of their warning mission. With the strategy of splitting, the probability of mission failure goes from 10 % (both on the same ship) to 1 % (each on a different ships). A relevant improvement! So Qui-Gon did the right thing.

What about the other cases given above?  The rules of probability say that if the probability of getting something is X %, the probability of NOT getting something is (100 – X) %. Consequently, the probability for a single man NOT being caught is (100 – 10) = 90 %. With the same rule of joint probability, we obtain that case 4 (the one happening in the movie) has probability

(90/100) * (90/100) =  81 %

and since probabilities must add up to 100 %, the remaining cases 2 and 3 are equivalent and get the rest, divided equally (100 – 81 – 1)/2 = 9 % each.

But wait. Cases 1, 2, 3 represent all those cases where their travel to the surface of the planet is discovered. The probability that any of these cases happen is, according to another law of probability, the sum of each individual probability

P(Case 1 or Case2 or Case 3) = P(Case 1) + P(Case 2) + P(Case 3) =

= 1 % + 9 % + 9 % = 19 %

If Qui-Gon and Obi-Wan were together, the probability of being found would have been 10 %, but that would also ruin the warning mission. So in this sense, Plinkett phrase of “increasing the probability of getting caught by 100 %” is almost correct: the probability that someone gets caught (either only one of them, or both) goes from 10 % (when both are on the same ship) to 19 % (when they board separate ships), but only a small fraction of that 19 % leads to a mission failure (both are caught), while in the former case, the 10 % probability of being found was also the probability of mission failure.

We can exaggerate this point with the following example. Suppose there are one million Jedi. According to our rules, if they all board the same ship, the probability of being found and therefore unable to carry the message is still 10 %. If each Jedi uses a different ship, we have one million ships with one Jedi each, and each of them has a 10 % probability of getting caught. The probability that not a single Jedi is caught is basically zero. It would be like throwing one million 10-faces dice and hoping not to get any “one”. However, the probability of all of them getting caught also goes to zero. It is like throwing one million 10-faces dice and getting a “one” on all of them.

As the number of Jedi increase, it is basically guaranteed that at least one of them is caught, so the probability of this event goes to 100 %, but as long as at least one Jedi is not caught, he will carry out his warning mission successfully. Since the probability of mission failure is also the probability of all Jedis being caught, and this probability goes to zero, splitting is a completely sensible strategy regardless of the number of Jedi… but maybe with one million Jedi the most sensible strategy would be to start some serious lightsabering, and roll the end credits after 3 minutes. Why didn’t Lucas think of it? It could not have been a worse movie !

By Stefano Borini | Posted in Movies, Probability | Comments Off

Simple risk management can change your life

January 10, 2011 – 12:25 am

It’s always a pleasing experience to change someone’s point of view by introducing him a new, simple concept. When this someone is a friend, the pleasure is even greater.

A few days ago I had a conversation with a friend, about how to deal with the uncertainties of life, in particular about employment, marriage and everything that can have a decisional impact on the future. It was during this conversation that I introduced him a simple trick I’ve been taught during project management courses: risk assessment and management.

Suppose you want to start a business, such as a popular social network site. Roughly, you will hire some people, write some code to provide the service, and have it run on a computer online, a so called server, and get people to use it. There are many things that can happen to hinder your path to success. Risk assessment is to evaluate what can go wrong, how badly it can go wrong, how frequently it can occur, and what effect has on your path to success (the establishment of a successful service), examples

  • While you develop your code, a thunderstorm occurs and fries your hard drive, thus losing everything you developed since the very beginning
  • The service provider that gives you online connectivity while you develop occasionally has downtimes, and you are unable to check online documentation
  • The server you put your service on occasionally has hiccups, rendering it unavailable for hours or even days
  • One of your team members gets sick for a couple of days
  • Issues in the offices require a temporary relocation of the programmer team to a different, already organized office for a period of time

There are many others, and they most likely depend on the kind of long-term objective you want to achieve. Once the risk factors (the WHAT can go wrong) are listed, you need to assign a level of impact (HOW BADLY it affects the achievement of your goal) and a frequency (HOW OFTEN it happens).  To do this, a useful diagram is the following

Risk Assessment diagram

We will explain the meaning of the zones later.

Outcome indicates the effect on your business. There are five levels:

  1. Disaster is utter destruction of any chance to achieve the goal.
  2. Major trouble is any problem that has a dramatic impact on the feasibility of the goal, hindering it into a very late shipping which can have important effects on future strategy
  3. Nuisance is anything that makes a non-trivial waste of productivity, a schedule delay, anything that is not strongly disruptive but still it is, well, a nuisance.
  4. Minor issue is something that, when occurs, it can be solved quickly with minor fuss nor impact on strategy
  5. No effect is anything that is irrelevant and represents no threat, not even minimal, for the goal.

Frequency is pretty self-explanatory. It indicates how often a given problem occurs. It’s not a matter of days or months. It’s a matter of estimating how frequently an issue comes through while you are working on your goal. The frequency goes from constant occurrence to no occurrence.

We now need to classify the problems given above according to the two scales given above. In some cases, you need to guess. In others, you probably have actual figures:

  • Fried hard drive with total code loss. Impact: Disaster. Frequency: Rare.
  • No internet connection. Impact: Nuisance. Frequency: Sometimes.
  • Hour-wide server hiccup: Impact: Major trouble. Frequency: Sometimes. It makes people look for more reliable services provided by competitors.
  • Sick team member. Impact: Minor issue. Frequency: Rare. We assume the team has another member able to cover the missing person for the brief interval. With a one-man team this would have a different impact.
  • Temporary Relocation: Impact: No effect. Frequency: Rare. We assume the programmers just need to plug their laptop and continue working.

Once the assessment is done, we lay down the points for each assessed entity on the graph. All the points falling in the “Safe Zone” are green. We ignore these issues because they represent something having no real impact on the goal to achieve.

All the points falling in the “Danger Zone” are red. These are serious problems that must be dealt with, or dire consequences may occur.

Finally, all the points falling on the “Control Zone” are yellow. These are issues we need to keep under observation, so that we know how to manage them when they occur.

Now that we concluded the assessment, we need to deal with the problems. Risk management is all about bringing the red points from the Danger Zone to the Control Zone, that is, on the line. You will probably think it makes sense to bring them in the “Safe zone”, but it’s a matter of cost/benefit balance. In most cases, having the situation “under control” is just the optimal compromise between costs and benefit. We have two possible strategies to control the risk: either we find strategies to reduce the frequency (moving the point along the direction indicated as 1), or we find strategies to reduce the impact (moving along 2).

For example, we could reduce the server downtime issue by having two service providers and balancing the service between the two, so that it’s unlikely that both are unavailable for hours at the same time. This approach reduces the frequency of the issue into an acceptable range. The data loss issue can be reduced by having a constant daily backup schedule, thus reducing the impact (that from total Disaster will probably skip the Major issue status directly into the Nuisance level, well into the Safe zone).

Once you learn how this works, you tend to apply it to daily life as well, ponder the frequency and impact that any event could have, and act accordingly to prevent uncomfortable situations. Example daily-life questions:

  • What if I lose my job in 3 years?
  • What if I have an accident and my children and partner are left alone?
  • What if I divorce?
  • What if the house needs unexpected renovations?
  • What if I break a finger ?
  • What if the car does not start tomorrow morning?

The major trick of risk assessment is to have reasonable, concrete estimates for frequency and impact of any issues in the list. In some topics, such as divorce, or loss of a relative, giving estimates often involves unpleasant considerations and emotional involvement. It is important to think with a clear, rational mind when estimating the occurrence of certain issues, eventually checking real statistics (the hard numbers), otherwise the assessment will be wrong and when one of the issues unfortunately occurs, there is going to be big trouble ahead.

By Stefano Borini | Posted in Risk Management | Comments Off

The Disposable Scientist

December 21, 2010 – 8:19 pm

I happened to read this very interesting article from The Economist with title “The disposable academic – Why doing a PhD is often a waste of time“. The article details in accurate terms the current condition of post-MasterDegree formation and professional development in the field of academic research. I invite you to read it, and I will sum up for editorial purposes. The point being made is that, simply speaking, obtaining a Ph.D. is increasingly becoming a waste of time. I agree in part, but more on feelings. If you look at the hard numbers, the point is clear.

Some time ago I registered a web domain, “disposablescientist”.com/org, with the idea of detailing witnesses and opinions about the situation of Ph.Ds and Postdocs around my network. The project never went online, not because I did not find anything to report, but because my time is limited and I decided to invest more time on ForTheScience, considering the good results I’m having. The situation the article on the Economist details is, indeed, very accurate. All around me, friends and colleagues detail a situation which is, unfortunately, rather sad. What are the root causes ? I think there are many.

The first cause I think is saturation. It is becoming increasingly difficult, if not even impossible, to obtain a tenure position in research. This is not only because it’s a very competitive environment among the peers, but it’s also because, not unlike a children’s game, all the available chairs are already occupied and the remaining players have no choice to stand up and leave the game. To increase the complexity, reductions in budget mean that when a professor retires, in most cases his position is closed. Kind of taking away the chair. Academia produces way more Ph.Ds than it’s able to absorb. Selection sooner or later must occur, and defending yourself during a job interview after you dedicated 15 years of your life studying obscure details of a DFT functional may be a problem. Considering the situation, the selection may well occur earlier. What is the lesser evil ? A 24 years old rejected at the Ph.D. admission or an unemployed, overspecialized 40 years old with children and old parents ? You decide.

The second important cause is money. This point is very broad, and encompasses a lot of sub-points which, in the end, all go back to the root cause of the hard cash. Research requires money: labs require instruments, tools, products, and disposals of wastes.

This stuff ain't gonna pop out of thin air

Disposing a one liter bottle of acetone (the same stuff in nail polish) costs many, many times more the price of the bottle of acetone itself. Until a few decades ago, people just threw the stuff down the drain, which is horrible and I am totally happy that things changed, but was definitely less expensive. I am not advocating the drain-option, I wouldn’t even think about it, but the point is that research has become more expensive. Returns, on the other hand, decreased, because we realized that the world, the body and the universe are very complex, and understanding them scales very badly. The more we dig in, the higher is the price to pay to evaluate small effects (or the network of effects) that are crucial for further investigation.

Materials, however, is just the tip of the iceberg. People are way more expensive. On this regard, other factors must be considered: need, and organizational scale. The need is related to the rapid shift of direction that scientific research may have today in response to political decisions, or new scientific findings. These shifts tend to favor “on-demand scientists” that may be useful for a given project, but may not be useful tomorrow when shift has changed, hence the per-project contractual situation. Specialization has become so extreme in science that it takes a lot of time (so money) to form competences. It is therefore difficult to recycle yourself towards other skills, not only because it may be time consuming, but because without sufficient reputation and proven experience in a field, it’s hard to stand out against the opponents in the race for money. In addition, if you consider that new skills can be taught and put into practice by cheap Ph.Ds, instead of full developed researchers, it makes economical sense to follow this strategy: a lot of disposable Ph.Ds.

Now, it may be objected that professor incomes are too high, and hard to maintain for 30 years of activity, and the budget must keep into account such high salaries. Maybe it’s a valid argument to say that tenure positions are overpaid, but we also have to remember that someone with the ambition of becoming a tenured professor had to invest a lot of time and take a lot of risks in specializing into a discipline with little or no “recycling value” in case of layoff or failure to obtain a tenured position, as pointed out before. To this, you must also weight in the stress of having basically no chance of settling down somewhere before you are 40, while in the meantime you basically live in crappy accommodation, constant relocation, awful services, facepalming immigration bureaucracy, general uncertainty and more.

Call it home

The scale issue is also interesting: in recent decades academia scaled up in size, and as any other company that grows, it became a streamlined, bureaucratic industry-like process that must achieve (at least on paper) three things:

  1. Produce scientific results (innovate)
  2. Produce new MD and Ph.D. (teach)
  3. Transfer knowledge to industry (concretize)

but at a lower level, remember, it’s all about money. Before detailing, let’s clarify one point: where does the money come from? The answer is “various sources”: grants from the European Community (in EU), university funds, student tuitions, State funds, Postdoc’s personal grants (such as Marie Curie Fellowships in its various forms), and industry spin-off and collaborations (eventually involving patents). Getting the money is not easy task, and requires experience, political influence, ideas, reputation, ability to provide what is needed at that particular moment, and a good dose of luck.

Given this premise, professors must dedicate all their experience and networking to “grant scouting”, applying for different ideas, in collaboration with different partners and hoping to get the money. Basically, professors today are overly busy funnels that move money from a financial agency to those who do the real research: Ph.D. candidates, who lack the experience to do research in the field, being therefore very inefficient but cheap proto-researchers; or Postdocs, who are more experienced but burned out. Postdocs realize their lack of goals, shattered dreams and rampant idiosyncrasies of the system, paired with advanced age, high stress, multiple concurrent tasks, long working hours, and no career perspective, apart from eventually becoming a politician/marketer/CEO of what, in the end, is a Ponzi/multilevel scheme. Research (in the form of scientific papers) is produced at the bottom of the food pyramid by Ph.Ds, and bubbles up to become reputation for those at the upper levels.

I wonder if this guy ever wrote a Part B Marie Curie IEF module

In addition to research, academia also performs teaching, which is also a source of revenue. The number of students a faculty has, how many graduates it produces and how many drop off, became measures of performance, which in turn decide where to send the money coming from the State or from tuition fees. This situation creates a race to the bottom, favoring overcrowded faculties with lax exams (a requirement further fueled by the degraded preparation level provided by primary and secondary education). After the Master’s Degree, those who continue for a Ph.Ds are seldom trained, either by their supervisor or the Postdocs, due to lack of available time. This provides a less than effective training. While being scientifically independent is a requirement for a Ph.D. formation, in some cases this situation just wastes time (and therefore money) just due to lack of communication: oftentimes, the solution is already within the group, but the people who know it are too busy with other things, mostly non-scientific ones.

In the end, is it worth it ?

So the big question stands. Is a Ph.D. worth the investment ? In my personal opinion, it is. Doing a Ph.D. teaches you autonomy, critical thinking, ability to overcome problems, ability to deal with unexpected circumstances, flexibility, and makes you understand research from within. Some of these may be learned also in industry, but industry tends to be, in the end, a different world, where speculation is hardly allowed. Research sometimes is pure speculation, in particular during a Ph.D., and it may end in something with a potential for innovation. Of course, it is important to choose an appropriate Ph.D. within an appropriate group. Doing a Ph.D. after some industrial experience is also, in my opinion, strongly useful: having a “production mentality” makes your Ph.D. more goal oriented, because you already know what is your research objective, and what you want to become. This holds regardless if you want a career in academic research or not.

By Stefano Borini | Posted in Opinion | Comments (1)

Augmented reality translation

December 17, 2010 – 4:04 pm

This is the coolest application I’ve ever seen. It does text recognition of the movie feed coming from the iPhone, then translates the text to another language on the fly (currently, Spanish to English and vice-versa) and renders the translated version on the movie itself, with proper coloring. The effect is so beautiful to be basically magical

… but it’s not. It’s technology, and as Arthur Clarke once said, any sufficiently advanced technology is indistinguishable from magic.

By Stefano Borini | Posted in iPhone, Software | Comments (3)

The Arsenic bacterium. A case of bad scientific communication?

December 4, 2010 – 1:54 am

As you probably heard in the news, two days ago everyone was ablaze for a mysterious announcement from NASA. Speculation started on how the new discovery would “impact the search for evidence of extraterrestrial life.” Someone found a habitable planet? Found a message with Seti@HOME? Discovered the primordial soup composition? The buzz resonated and amplified at every new passage, like in a Chinese whisperers game, speculations were born and distorted, sometimes by the lack of scientific knowledge of the message carriers, sometimes by a deliberate “science-fiction twist”. Now that the announcement has been made, it’s a big disappointment, a delusion… you name it.

I’d like to propose to make a step back and see the thing from a bit of distance. Let’s analyze the facts. Life (from the tiny bacterium to a giant whale) generally uses six fundamental elements: Carbon, Hydrogen, Oxygen, Nitrogen, Sulfur, and Phosphorus. From these six elements, a large majority of the fundamental building blocks of life are created: DNA, Acetil CoA (a transporter molecule), ATP (for energy storage) and many, many others. Yes, some other elements are used, but they are in a tiny amount, and although this tiny amount is essential for life, it is really a pinch.

On this planet, life managed to conquer places that are considered “extreme”. There’s a class of bacteria called “Extremophiles” which collects all those living under really unexpected conditions. For example, there are bacteria living in almost boiling water, or in very high concentration of acids. The “extreme” conditions are, however, in the eye of the beholder. For these bacteria, their condition is just the norm. For us, living in a scorching boiling hot spring, or in a poisonous acid well is “extreme”, as it would be extreme for them to live at 25 degrees of temperature in a neutral pH. In fact, they are well adapted to their niche. With the recent rise of techniques for biological analysis and sequencing, we are exploring more and more of these niches, and discover more and more bacteria (or communities of bacteria) well adapted to these niches.

There is a lake in California called Mono Lake. It is one of those extreme places when it comes to conditions: high saline concentration and alcalinity makes it probably a very unpleasant place to take a bath. Fishes agree, and quickly give up and die, so the only thing that lives in that water are some shrimps, algae and bacteria. To complete the nice picture, it has one of the highest concentration of Arsenic, which is a poisonous substance to us, and in general to all organisms. As said before, any bacterium living in extreme conditions for us is classified as “extremophile”. GFAJ-1, the little bug announced by NASA rightfully belongs to the category, since it lives in the waters of the Mono Lake.

The bacterium has proven really, really tough with respect to Arsenic tolerance. Grown in a lab with even higher concentrations of Arsenic and no Phosphorus at all, the bug decided to switch to Arsenic when it comes to inclusion in its fundamental components, replacing Phosphorus in DNA, ATP and so on. In practice, faced with a lack of a really important element such as Phosphorus, he said “screw that, I’m using Arsenic, it’s good enough for me”. This is really a novelty, but it is not revolutionary. It is just a very important finding, being the first time where life uses completely unusual strategies to continue thriving.

Summing up, the content of this discovery is:

  1. for the first time, a different element (Arsenic) was experimentally seen as a substitute for another element (Phosphorus) to build fundamental biologically important molecules in-vivo.
  2. there’s no number 2

Seriously, it’s a great finding, but that’s what the facts tell us. No aliens, no totally different life form on earth, no implications for space research… maybe just a little: we should not be so strict in assuming Phosphorus as so fundamental for life, given that it can be successfully replaced. We also should realize how resilient and adaptable is life when faced with harsh conditions.

Excessive hype and broken dreams

The announcement of the conference at NASA landslided a serious amount of speculation and hype. Maybe it was intentional, maybe it was just a mass craze triggered by unintentional circumstances. I cannot say. What is clear is a good amount of blog posts (1, 2, 3, 4, 5) pointed out the excessive hype, and the sheer disappointment that followed. This in turn fuels news fatigue. As I already pointed out in a previous post, this news fatigue in turns fuels misunderstanding, detachment and apathy towards new discoveries. We all want NASA to tell us: “listen, we got that call from ET” but seriously, it’s not how science normally works.

So what’s the correct recipe ? What do you have to do and understand when you read or hear some scientific news, and how much weight do you have to apply? In my next post I will tell you simple rules on how to read scientific  news with a critical and realistic eye, directly from my experience on scientific papers. Stay tuned.

Edit: I decided to postpone the continuation of the article for now. I already have a draft, but I am quite busy at the moment, and I have a very long queue of scheduled articles already. I may insert it in the queue, however.

By Stefano Borini | Posted in Astrobiology, Authoring, Biology, Dissemination | Tagged , , | Comments (4)

base32 encoding in javascript

November 30, 2010 – 5:47 pm

I had to perform base32 encoding in javascript, and I found nothing ready for the task, so I started cramming out code, and here it is

var baseenc = baseenc || {};

baseenc.b32encode = function(s) {
 /* encodes a string s to base32 and returns the encoded string */
 var alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZ234567";

 var parts = [];
 var quanta= Math.floor((s.length / 5));
 var leftover = s.length % 5;

 if (leftover != 0) {
  for (var i = 0; i < (5-leftover); i++) { s += '\x00'; }
  quanta += 1;
 }

 for (i = 0; i < quanta; i++) {
  parts.push(alphabet.charAt(s.charCodeAt(i*5) >> 3));
  parts.push(alphabet.charAt( ((s.charCodeAt(i*5) & 0x07) << 2)
                               | (s.charCodeAt(i*5+1) >> 6)));
  parts.push(alphabet.charAt( ((s.charCodeAt(i*5+1) & 0x3F) >> 1) ));
  parts.push(alphabet.charAt( ((s.charCodeAt(i*5+1) & 0x01) << 4)
                               | (s.charCodeAt(i*5+2) >> 4)));
  parts.push(alphabet.charAt( ((s.charCodeAt(i*5+2) & 0x0F) << 1)
                               | (s.charCodeAt(i*5+3) >> 7)));
  parts.push(alphabet.charAt( ((s.charCodeAt(i*5+3) & 0x7F) >> 2)));
  parts.push(alphabet.charAt( ((s.charCodeAt(i*5+3) & 0x03) << 3)
                               | (s.charCodeAt(i*5+4) >> 5)));
  parts.push(alphabet.charAt( ((s.charCodeAt(i*5+4) & 0x1F) )));
 }

 var replace = 0;
 if (leftover == 1) replace = 6;
 else if (leftover == 2) replace = 4;
 else if (leftover == 3) replace = 3;
 else if (leftover == 4) replace = 1;

 for (i = 0; i < replace; i++) parts.pop();
 for (i = 0; i < replace; i++) parts.push("=");

 return parts.join("");
}

I release this code under the obnoxiously named but quite appropriate WTFPL license. I left the decode, as it is commonly said, as an exercise for the reader. Seriously, I didn’t need decode, so I did not invest time into it. Sorry, maybe in another post !

By Stefano Borini | Posted in JavaScript | Comments Off

Exploring Mandelbrot parameter space – part 2

November 15, 2010 – 2:17 am

In the previous post, we saw that graphing the Mandelbrot starting point contains fractal features as well. We want to plot these features, and at the same time increase resolution but reducing computational cost.

The first thing to note is that apparently, the plot is symmetric (but I don’t have any strict proof of it) so technically only 1/4th of it can be computed, at least during the exploratory phase. The second optimization is that we are going to plot either a white pixel (if the corresponding Mandelbrot contains at least one white pixel) or a black pixel (the Mandelbrot contains no white pixels). This dramatically reduces the time needed, because when plotting each Mandelbrot, we can exit the generator as soon as a white pixel is found. White pixels are computationally expensive because the iterative procedure for that pixel must run up to the end, hence very whitey Mandelbrots, which are the majority, require a lot of these “up-to-end” iterations. This wastes a lot of time to compute something we know as soon as the first white pixel is found. On the other hand, Mandelbrots containing no white pixels have their iteration loop cut short for basically every pixel, making them faster to compute even if we have to run on each and every pixel of the Mandelbrot.

After these improvements, I ran two calculations. One using 100×100 Mandelbrots (on the left), another using smaller, 10×10 Mandelbrots (on the right)

Apparently, the best visual results of the fractalization are obtained with smaller Mandelbrots. I am not really surprised, we already saw a similar situation in the previous post, where I plotted according to the number of white points. I have the feeling that increasing the resolution (the plots you see above are from 1000×1000 images) we would see better fractalization in the 100×100 case as well, but I would not bet on it. In any case, the 10×10 Mandelbrots are sufficient for our purpose, and they are much faster to compute. It’s time to increase the resolution up to a staggering 900 Megapixel (30.000 x 30.000), on the full set, and see what happens. What you see took 10 full days to compute. I am not posting the full image, just a small low-resolution screen grab. The image is so big that is unreadable on Firefox, hangs Gimp and puts Preview.app in a very unpleasant situation. Here is the overview

Overview

As said previously, the Set appears to be symmetric. The fractal features we observe on the border are unique, in particular due to the existence of compartments with common behavior. Please note that all the terms I will use along this analysis are invented by me. Nothing of what you read is formal or agreed nomenclature. I will start from the left hand side of the border, and walk up to the top.

First, we have what I call a spike, a long, narrow line running horizontally and stretching away from the central bulge. This feature is found in the Mandelbrot set as well.

Mandelbrot whiskers

Spike and Whiskers

If you click the image and zoom in, you will notice a very interesting feature which also can be found in the Mandelbrot Set: lateral whiskers. These curved features intercept the spike in very bright areas. In traditional Mandelbrot, these areas are rich in self-similarity, as you can appreciate from this magnificent Mandelbrot image from Wikipedia. I don’t know if similar self-similarity can be found in this Set as well, and the current resolution does not allow to infer any statement. According to what I understand from this answer Sam Nead gave at Math StackExchange, self-similarity is connected to what I assume is a mathematical property called renormalization. Due to my lack of expertise, I cannot explore this any further, but I encourage anyone competent on fractals to add comments, fix my mistakes and provide enlightenment on this regard.

Prickly pears zone

The next interesting feature is what I call the prickly pear zone. This zone has very sharp convergence points, and a large number of buds with similar shapes. The central zones look slightly polygonal in nature. I call it prickly pears due to its resemblance with the Opuntia (indian fig) plant, a cactus with characteristic fruits.

Prickly pears (Indian fig)

The sharp contact points recall similar features found in the Mandelbrot set, but on the other hand, the polygonal-like structure is not found. Mandelbrot, at least in its full, unzoomed representation, contains circular structures and the main cardioid.

The third interesting feature is the tree foliage.

Tree foliage

which appears as a relevant change from the repetitive, sharp-edged prickly pears area. On this regard, however, nothing beats the nervous, highly featured, strongly self-similar and spiraling behavior of the fuzzy worm. This area of the plot is incredibly featured and a real pleasure to zoom in.

Fuzzy worm

After the fuzzy worm, we don’t see any more spectacular features. The border becomes first rocky, then smooth then rocky again, and it’s finally closed with a hole.

Mandelbrot final part

As usual, I cannot really say anything about the mathematical content of the task I performed and the features of the resulting fractal. However, it has been an incredibly interesting and fun exploration into a fascinating discipline. I hope you enjoyed this exploration, and feel free to leave comments if you want to add information or fix mistakes.

By Stefano Borini | Posted in Fractals | Comments (1)

Exploring Mandelbrot parameter space – part 1

November 1, 2010 – 4:19 pm

Some time ago, I presented an interesting python code able to draw the so-called Mandelbrot set, a fractal image with intriguing properties. Recently, Benoit Mandelbrot passed away. I want to pay homage to his work by digging into more details of his eponymous image.

In the previous post, we observed that a parameter is crucial for the resulting image: the “starting point”, described in my code as (a, b) or, in the second part of the post, z0. In traditional Mandelbrot, this starting point has coordinates z0 = (0.0 , 0.0), and with this input, the Mandelbrot set is obtained.

After this first evaluation, I went further on by showing how the Mandelbrot Set changes when this point z0 is modified from the original value (0.0, 0.0). The animations I provided show what happens to the Set when choosing different z0 points, such as (1.0, 0.0) or (-2.0, 0.0), for example. The first animation showed plots with z0 = (a, 0.0), with a going from -3.0 to +3.0 with a step 0.1. In other words, I produced a Mandelbrot image for z0 = (-3.0, 0.0), another one for z0 = (-2.9, 0.0) and so on, until z0 = (3.0, 0.0). In the second animated picture, I did the same for points like z0 = (0.0, a) with a on the same interval and step. Finally, the last animation shows how the Mandelbrot behaves with starting points (a, a), again with the same interval.

This exploration of the bond between a possible value for z0 and the resulting Mandelbrot has interesting properties. Let’s try to see visually what’s going on. Imagine there’s a plane of possible values of z0

The plane of the possible values of z0

For each point on this plane, there’s an associated Mandelbrot Set image. The traditional Mandelbrot is associated to the origin of this plane. With the experiments I did by changing the starting point, I moved along the x axis in the first animated picture, along the y axis in the second animated picture. For presentation purposes, I could choose to stick the actual Mandelbrot images on this plane, imagining that the lower left corner of each image indicates the z0 point that image comes from

Mandelbrot sets for particular points of z0

Mandelbrot sets for particular points of z0

As I already stated, the lower left corner “points to” the z0 point. You see, along the x axis, images for the cases (-2, 0), (-1, 0), (0,0), (1,0), (2,0). Along the y axis, you see (0, -2), (0, -1), (0,0), (0,1), (0,2). These are only a few of the points you could actually choose. Any point of the plane has an associated Mandelbrot.

My plan for this post, is to explore the plane, that is, to see how the Mandelbrot image changes as I select points like (0.34, 0.12), among others. In order to achieve this, I need to do exactly what I did above, but with smaller images with respect to the size of the “graph paper” image. The results are quite interesting

z0 plane. Zoom out

z0 plane. Zoom out

The image above shows the z0 plane. It is built with the same concept given for the very simple case above: many Mandelbrot images, whose corner refers to the actual point on the plane. To demonstrate you this is the case, if I zoom in what I see is this

z0 plane zoom in

z0 plane zoom in

A collection of very tiny, very low resolution Mandelbrots, each of them associated to a specific point of the z0 plane. I had to keep the resolution of each individual Mandelbrot to very, very small. What I did takes a lot of computer time.

The interesting fact about this plot is that it contains fractal features as well! Looking carefully on the left, and playing with the zoom, I obtained this image

z0 features

z0 features

I suggest you click on the image (so to get a bigger version) and observe it from afar. The ramifications typical of fractals will be evident. Every small white dot in the above image is a “mini Mandelbrot” image, whose white points allow us to see something, but not very clearly. Can we improve the situation? Let’s see.

I developed a new program. Instead of placing tiny Mandelbrot images like post-it notes on a board, for each point of the z0 plane I generate the corresponding Mandelbrot, then I count the number of white points it has, and I color the pixel of the z0 plane of a different shade, depending on the amount of “whiteness” showed by the corresponding Mandelbrot. The result ?

Click to enlarge. The above image was created from the number of totally white pixels (scaled from 0 to 255 against the maximum) in a series of 100×100 Mandelbrots, one for each pixel. Not as fancy as I hoped for. I tried to play with color balance to no avail. I think there are two problems. First: the small resolution (1000×1000 is not much). Second, the poor averaging strategy, leading to poor contrast. This plot required 48 hours of computation.

Trying to get better plots requires a lot of computational time to increase the resolution, but unfortunately I only have a laptop. This kind of problem can run parallel very efficiently, but even if I put my second core to work, I won’t get very far. The next step is therefore to get a better plot at a reduced cost, eventually accepting some compromises. Stay tuned.

By Stefano Borini | Posted in Fractals | Tagged | Comments Off

An initiative to promote science through a symbol: tulips

October 25, 2010 – 10:30 am

After I came back from TAM 2010, I was waiting for my train to bring me home. I started thinking that there is a potential Public Relations issue with scientific proponents and skeptics around, and it’s never good to have a Public Relations issue. I feel there’s a need to change this, and a need to do good to promote critical thinking and scientific research, raise awareness about science while promoting well being.

The stream of thoughts focused me to the following needs:

  1. Reduce the perceived vision of “dicks” and instead promote an image of carefulness, long term planning, passion and love for beauty
  2. Do some activity requiring small expertise, promoting a gratifying, beautiful result
  3. Do some activity whose apex is during a relatively narrow period of time, so that there’s a particular period of the year where things happen and people see the result
  4. Something symbolic must be used. The symbol must be nice, and increase the level of happiness. A symbol that uses nature, and focuses on growth through human contribution
  5. Some good, concrete action must be done as a concurrent activity
  6. It must act as a constant reminder of your commitment
  7. It must influence the life of others, without bragging about how good and kind you are, because bragging may be perceived as impolite

After a while, I decided to go with the following

Yep, that’s a bag of tulip bulbs. Cheap, and full of beauty.

Once a year I receive the royalties from the book I published together with other, much greater than me people. I decided to put these money to a more productive goal than buying a new hard drive, and I donated them: Wikipedia and Doctors Without Borders. (yep, sorry I have to brag just for the argument)

After my train of thoughts, this year I decided to associate this donation to gardening. I’ve never grown anything in my own life, and I have nobody around able to give me hints, so it’s definitely a challenge for me. The idea works like this: I plant the tulip bulbs in October. When in Spring I get a nice tulip, I donate the flower to a person I know, and at the same time I donate to a science-oriented no-profit organization.

Do you want to follow me in this idea? good. Here are the general rules:

  1. Plant the tulips. Here are instructions and some Q/A. They must be put into a vase during the Fall (that is, now, if you live in the northern hemisphere), and stay outside in a sunny, dry place.
  2. Send me an email or twitter message to inform me you planted a bulb. My email is in the About Me page.
  3. Tulips bloom in Spring. If you are a geek, you can try with a time lapse of the growth, but it’s enough if you take a picture of the flower in its vase. After this, either cut or move the flower in a small vase. Give the flower to a person you know, and don’t tell him/her what’s about. The point is to make someone happy with a simple act of kindness. If him/her is interested to know more about what’s going on, he/she will find out. You have to know the person because giving something to strangers for no reason is likely to make them uncomfortable, rather than happy. Go for someone who trusts you.
  4. Donate something for correct scientific dissemination, critical thinking, health care, environment protection and cleanup, or anything else that improves quality of life through good science and encourages love for science and the scientific method. Your choice. It could be money, it could be some relevant amount of your time. If you choose to donate, choose a non-profit that promotes science. For example, in Italy there’s Telethon for muscular dystrophy, but you can also go for cancer research and assistance (like Cancer Research UK, for example), STD prevention… your call. Try, if possible, to promote organizations or activities based in your own country.
  5. Send me the picture of the flower. If the idea catches on, I will try to organize a website to publish the pictures.

Why a tulip you may ask? I’m in the Netherlands, tulips are beautiful, simple, easy to grow, and it’s the right period of the year. It seems a natural choice.

Have fun!

By Stefano Borini | Posted in Charity, Gardening, Opinion | Tagged , | Comments (2)

TAMLondon 2010 remarks and comments – Part 4 of n

October 21, 2010 – 10:44 pm

Notable talks

Susan Blackmore presented an excerpt of her first years of  research on paranormal. She shows intriguing statistics on paranormal evaluation, describes some techniques used for cold reading. A consistent part of her talk was relative to physiological explanations of different phenomena our minds perceive as paranormal or mystic. More specifically, she touches arguments such as sensory deprivation, sleep paralysis, the structure of the visual cortex and the Olaf Blanke experiments (see also here). All these techniques and experimental setups provide good explanations for perceived paranormal events, such as ghost appearances, alien abductions, light tunnels and out-of-body experiences.

Richard Dawkins held a thoughtful and poised talk on the preciousness of scientific literature to be considered classic education, while scouting the broad number of scientific disciplines describing the mechanism of our world and life in particular. Dawkins presents a talk both source and inspiration of great quotes, enticing the audience with a profound philosophical vision: “Science is uplifting, inspirational, poetry of reality“, “If more of our political masters understood statistics, the world would be a better place“. The humbling tree of life shows our position in the evolutionary development, a twig of a deeply branched bush covering animals, plants, fungi, bacteria.

In addition to its high philosophical tone, the talk presents real case scientific scenarios for disparate arguments, such as: flight stability strategies in dinosaurs and birds to reach the optimal compromise between stability and ease of flight versus instability and high manoeuvrability; Computational evolutionary optimization of windmills and fuel injectors nozzles to achieve the best performance; and evolution of human language, maybe the best platform to explain and demonstrate evolutionary concepts to the general public.

Cory Doctorow presented a talk about copyright and the historical background of Intellectual Property (in the early days known as creative monopoly). In his words, Doctorow says that “yesterday’s pirates are today’s admirals”, referring to how past piracy allowed development of companies that are now brands with worldwide recognition and technologies whose revolutionary nature changed our lifestyle. Doctorow presents an example of copyright stifling innovation for databases: while in UK any collection of data was protected under IPR laws, in US no such protection was present, allowing the creation of an incredible amount of databases, and consequently better associated services.

Karen James talked about the Beagle Project, finalized at rebuilding a replica of the HMS Beagle and set sail along Darwin’s travel to promote scientific dissemination, improve public engagement with science and encourage young people to become scientists. As I am starting a sailing course as a crew member next spring, this project hit me on the sweet spot, dreaming to be part of this incredible travel.

A board discussion held by Paula Kirby focused on skeptical activism. Most of my comments were presented in my previous post. The discussion produced interesting resources worth checking out, such as “Sense about Science” and the evidence-based policy initiative from the “Nightingale collaboration“. These initiatives are all UK-centric. I personally asked a question about similar initiatives and strategic networking with other European countries. Let’s see if it grows into something.

The talk given by Marcus Chown goes through recent discoveries to provide a mindblowing list of scientific oddities. Did you know that the Sun uses a very inefficient nuclear reaction to produce energy, and it’s due to its inefficiency that it lasts enough for us to be here ? What about the fact that if you are at the ground floor you age slower than being at the first floor, and very accurate clocks are able to measure this difference even between two stair steps ? Did you know that a small percent of the white noise you see on an untuned TV is actually the echo of the Big Bang ? Marcus provides a brilliant and really pleasant talk, both conceptually and visually.

The Big Bang. Sorta.

Finally, JREF president D. J. Grothe and P. Z. Myers gave two different talks with overlap and disagreements, concerns skeptic activism and atheism. Grothe points at the need for skeptics not to be “dicks” in their positions, and have a tender heart with a tough mind. I share his point of view, and I wrote about the same topic some weeks ago. P. Z. Myers on the other hand claims a different, more intransigent attitute, in particular when it concerns skepticism on religious issues, i.e. atheism “Do be a dick. Be the best dick you can be.” or, in his words “do be a Richard”.

Myers’ point is supported by his observation that by their very nature, atheists are “honest, assertive, humorous, angry, critical, militant, scientific, uncompromising, rude”. I don’t share his point of view on some of these terms, but I do agree that for some people, even the pure questioning of a belief (regardless of it being religious or not) is automatically classified as rude, no matter how positive assertiveness and smiles you can put in presenting your evidence. I already wrote about the case: haters gonna hate, stick their fingers in their ears and start singing. No matter how much evidence is carried, people don’t like to be proven wrong, and they will continue using their “miracle cure” or “magic bracelet”. In this sense, Myers says that when there’s basically no possible way of not being classified as a dick, the solution is to try to be the best dick possible.

A comment on Grothe/Myers Talks

My opinion on the point raised by Myers is different: you should choose your goals wisely, and work for something, not against something, with patience, passion, and righteousness. While it’s clear that immediate and firm stance must be taken against people selling poison or sugar for medicine, or scamming old people and grieving mothers, or giving a free pass to intolerance and discrimination, the long term goal for skeptics should be to increase scientific education, promote correct popularization of science, explain with a clear and accessible language, and dispel fear and myths. These, in my opinion, are the productive long terms goals skeptics should focus on.

This is my last post regarding TAM London 2010. I want to thank the speakers, the audience and the organizers for the fantastic experience.

In an upcoming post, in a couple of days, I will introduce a symbolic action I am going to take to promote goodwill and personal growth, while making someone happy in the meantime. My hope is that other people will appreciate this idea, so that it becomes a pleasant activity and potentially a way to promote skepticism and concrete actions towards better science. After this upcoming post, it will be back to science at ForTheScience.

By Stefano Borini | Posted in Meetings, Opinion | Tagged | Comments Off