Floating point numbers

Sometimes we all need to deal with the floating point weights. A normalized values between 0.0 and 1.0.

Every time I deal with it I think about normalizing it to percents 0%-100% instead of 0.0-1.0. Why?

Because human beings, including myself, always think linear. I think: 0-100 has a better precision than 0-1. Our floating point numbers are discrete. There are more possible values between 0 and 100 than between 0 and 1. So our calculations are 100 times more precise with the percents. Right?

Wrong. Absolutely wrong. The IEEE-754 floating point format does not put numbers uniformly. This is not an even distribution. The distance between two floating point values depends on the module. For example, if we use a 32-bit float, the closest number to 100.0 is (100.0 + 2 ^ (-17)) . The closest number to 1.0 is (1.0 + 2 ^ (-23)).

The density around 1.0 is 64 times higher than the density around 100. The distance between closest floating numbers grows exponentially.

Even though there is more numbers between 0 and 100, the precision for a reasonable numbers like 50,60 etc is better when they are normalized to 1.

The moral is - don't be afraid to normalize. Normalization never cause loosing precision. More than that, if you are doing extensive calculations with large floating point numbers, it makes sense to normalize them, make calculations, and de-normalize back at the end. There is a big chance that the accumulated error is lower in this case.

Still idling

No, I didn't quit Puppeteer.

I'll get back to it as soon as I get rid of the current contracts.

I'm developing for iPhone now. Honestly speaking I don't like Objective C. IMHO, this is a poorly structured and unsafe language. I don't like Mac either. And particulary I don't like XCode.

The only thing I like about this project: I earn some money and I earn some XP to port Puppeteer to iPhone later.

This is all I have to say: The Puppeteer is still alive, but I need to spend some time on other tasks...

Puppeteer demo

I finally found some time to upload the Puppeteer setup files.

This is the Puppeteer release number zero. I call it demo-09.10. The puppeteer versions will be called after the year and the month they will be released. For example, if I will make a release on February 2010, it will be called puppeteer-10.02.

This demo includes the asset manager itself, a simple skeleton editor and a simple animation editor.

All the scenes, controllers and others are not ready yet, so they are excluded from the demo.

Download the Asset Manager demo.

EA

EA is still laying people off. They don't do huge events like resigning 200 employees or closing huge studios. They are just laying off small groups of people. One of my friends Yulay has been resigned recently.

What is wrong with the company? It looks like Titanic now. Huge, leaking and out of control. The stocks continue to go down. I think they are doing worse then the other companies in the industry.

Well, may be I'm wrong. I don't exactly know how other companies are doing :)

My own impression is - this is all about management. EA has too many managers. And the situation is getting worse because they resign artists and software engineers. It doesn't look like they are resigning managers. Again, may be I'm wrong. I don't see the whole picture. This is just how it looks from my point of view.

Once when I was working there, I was looking to the intranet to find who are the managers of my manager. I was really surprised by the number of managers above my manager. There were the chain of 10. John Riccitiello was a manager of a manager of a manager of a manager.. oh I'm getting tired saying it 10 times .. of my manager.

Reflecting to the Soviet army.. Why Soviet army? Because I don't know much about other armies. Considering I'm a soldier. My nearest boss is a sergeant. Then a master-sergeant, then a lieutenant or a captain, then a major or a colonel, then a major-general or a lieutenant-general, then a colonel general, then a marshal, then the commander-in-chief. Eight levels.

Do you know how big was the Soviet Army? About 10 million. And it had less management levels than EA. What are all these people doing?

Well, I promised not to discuss EA here. Sorry.

Idling

I'm taking a two-week break.

I have signed a short contract - developing a pretty small and simple application for iPhone.

Just want to earn some money and take a short break from the Puppeteer.

I'll be back as soon as the contract will be done.

Compiling assets performance

I have spent too much time trying to improve the compiling performance. That is enough now, I am quitting it. The priority of it is quite low anyway. I need to work on scenes and actors, I need to develop the runtime..

Well, you all know how this happens sometimes. You are so much into the problem that you stop seeing the whole picture for some time. You just can not stop doing it because it is not done. On a rational level you understand that there are more important things to do, but you think: duh, just one more day.
Etc.

This is what happened to me. The good news is that the Asset Manager is pretty much done. I will allocate "just one more day" tomorrow to make an installer, and create something downloadable. This is not a release yet, this is just a preview that gives you a feeling of what Puppeteer Asset Manager is.

The compiling performance is still an issue, but it is not the end of the world if I fix it after the release.

Compiling in Puppeteer is a process of creating runtime files - the files that are used in the game. Assets in the Asset Manager contain some data that is not needed in the game. For example, asset names, channel names for animations, bone names for skeletons. There is no need to waste the game memory for it. Another thing to consider is compressing animations. The Asset Manager uses the raw animation data. It is a good idea to compress it before using in a game.

Thus, Puppeteer uses different asset data files in the Asset Manager and in the runtime. As I already mentioned, converting the Asset Manager assets to the runtime files is called compiling in the Puppeteer.

Compiling in Puppeteer is quite similar to C/C++ compiling. Puppeteer compiler creates a kind of obj file for every asset - the extension is ppo. Then it "links" them to a partition files - extention ppp. It also creates a runtime file - ppr. Ppr file has a general information about asset types, partition dependencies, etc. Ppp and ppr files are the files that are used in the game.

Initially I created a very simple single thread compiler, that created ppo files one by one, and then generates all the ppp files. Compile then link approach.

The performance was just fine on a stress test project - 21-22 minutes. I thought making it multithreaded could reduce the time to 7-8 minutes. That didn't happen. All I could achieve in 4 threads was 15-16 minutes.

Linking is a bottleneck. I am using the next algorithm now:

- Collect all the dirty assets to an array.
- Based on the dirty assets array, collect all the dirty partitions to an array.
- Start 4 threads. Every thread function:

1. Check the partition array for a partition to compile. If it is not empty take one partition, remove it from the array and goto 3.
2. Check if the assets array is not empty. If it is not take an asset, remove it from the array, compile it and goto 1. If the array is empty - exit.
3. Go through all assets of the partition. For every asset. If it is in the dirty list - compile it and add to ppp file. If it is not in the dirty list - just add the ppo file to the ppp file. If another thread is compiling it right now - compile another asset and check again after it. If there is no another asset to compile, just sleep a few milliseconds. After the partition is compiled, goto 1.

For example suppose we have 1 partition with 1000 assets. In this architecture the first thread takes the partition, the other 3 threads are compiling the assets. So linking and compiling happens at the same time. If a needed file is not ready, the linker stops and helps with the compiling.

The problem is that if we change just one asset, the linker still has to add all 1000. Compiling compile their one asset and exit, and the rest happens in one thread. Considering that every asset is 30k in my stress test - this is generating 30Mb file from 1000 other files. That definitely takes some time.

Incremental linker is the solution, but not now. I have spent enough time, and after all I do not expect 64000 assets projects in the nearest future :-)

Another container code to share

This time it is a C# red-black binary tree.

I wrote it when I considered reimplementing TreeView and ListView standard controls. Their performance was a bottleneck. I thought that making similar controls that use less memory allocations and better sorting algorithms could solve it. However after I reduced the number of posted messages, the performance became pretty decent.

Thus the tree is not used in the Puppeteer at this moment in time, but still it is a pretty good red-black tree. The performance was improtant, so I tried to avoid memory allocations as much as possible.