Thursday, December 16, 2010

In the beginning (A Story of Lorito and Parrot)

In the beginning...

...there was Parrot.  And it was magical.  But it was slow and would not stand still, so it was hard to target.  Then there were calls amongst those near Parrot for speed and efficiency.  Thus, Lorito was imagined.  It was to be leaner, faster and more stable, but still be able to handle almost all the magic of Parrot.  Ideas were thrown around, and much hand waving ensued.  Then, an idea of limiting the core to 20 opcodes was thrown out.  This piqued my interest, so new code was written, stubbs were created, and a project on github created.  Code was written fast and furious and a vision appeared.  I began to dig at the surface of the issues that Parrot exhibited today.  Implementation details began to form around avoiding the mistakes of the past, as they were described to me.  Soon, a mantra was formed: Less Magic == More Magic.  Simplicity and efficiency can be optimized later, but complexity today remains tomorrow.

First, I made the opcodes single sized.  8 bytes, and only 8 bytes ever.  1 for the opcode, 3 for the 3 arguments (destination, source 1 and 2), and 4 bytes for the immediate value.  Quickly, I meet resistance.  "We need far more than 255 registers" was the cry I heard.  I took note, contemplated doubling opcode size, which could afford 4k of registers.  Then I heard the reason why they needed more registers.  "Because that's what's produced today."  When I dug the surface of the claim, it appeared that there is no register allocation scheme.  That many were used because they could be.  I said to myself "I have a hard time believing that at one point in so many programs, that a function call would need to track more than 255 registers at one time."  So the decision stayed.

Then I said that all PMCs are pointers and have no inherent structure.  There would be no way to ask, politely or otherwise, the PMC itself what it looked like.  The PMC could have a gate keeper, but forcing that seems like the wrong solution to the problem.  Why, I was asked.  That will make them flexible, I respond.  I have removed no functionality, no features lost.  Everything that has been done before can still be done.  Do we complain that machines have blocks of memory that we access by address and not by name?  But now, you can manipulate, store and load any data of any size and requirements into a PMC as though it is merely memory, because it is.  Since the PMC is always the same size, and it is the only thing that points to it's memory, being able to rearrange after garbage has been collected for the time is trivial.

Then, upon the advice of elders, I stopped treating PMCs with a level of reverence that objects do not enjoy. They are the same, I declared.  No difference.  Objects are PMCs, and PMCs are objects.  No more tables of v, they are all methods to be lookuped and called.  Perhaps, I whispered to myself, we still need some special functions.  Simple, I replied, for the information you can ask of all PMCs, make those special functions special.  But do not let others interfere, and make their numbers small.

I surveyed my objects around, and saw my scatterings.  Registers put into boxes, segments of consts, datas and code.  In inspiration, I promoted them all to the same level.  Special PMCs, yes, but PMCs none the less.  To be interacted with in an identical matter as all other objects.

Then I noticed the registers that spoke words.  What use are they as special?  But, how do we efficiently do lookups on objects without them?  Then someone spoke of symbols.  What are these symbols, I asked.  He showed me a world where comparison was cheap of large volumes text.  Symbols become the new register and we gain performance for lookuped objects and regular manipulation of these strings become objects, like everything else.

I create a container that contained everything Lorito needed to know about what it was doing.  It had the registers, the code it's working on, the arguments it has and returns, on the constants it has access too.  It has no requirement to return where it came from, so that will relieve some concerns about being able to do magic called CPS.  This context object then leaves the details of how the operations actually take place inside an interpreter for Lorito.  Later, after a braindumping, the idea was reinforced by those in attendance.  Thankfully, brains intact.

How can you call methods of objects, some asked.  Here's an idea, called P&W, another one told me.  Interesting, it was, but perhaps was still too complex since it dictated inheritance.  What if there is no need for inheritance?  What if inheritance is complex?  Let the object deal with it all.  Make a special operation called lookup.  When it's called, call the object's special lookup method and pass it whatever object it handed us in the beginning.  It shall let everyone talk to everyone, and allow all any possible complexities to be possible.

Lorito has limited operations, and the instructions are limited in size.  The way to call methods will have to change.  No longer can we pass all of the arguments in one operation.  So a context is created to contain these arguments and other details, and the arguments are added to this in separate operations, not unlike a stack of trays.  Then, when the method is done, it can return any information in the same list.  But this method is not without controversy.  Efficiency, speed, compatibility are the concerns I hear.  I cannot speak to any of the three, but I believed it was my only choice with my decisions.  But the calls to those not inside the sphere of Lorito, they can be simplified.  These outside functions can be treated inside Lorito exactly like Lorito methods.  Then, once this function is called, it is passed the context I already have and it can do it's magic.  A nice, simple way of interacting with those that are not Lorito.  Sadly, the concerns of others cannot be satisfied for some time, but I persevere.

Parrot has a difficult time talking to itself.  It can ask itself a question and wait for an answer, what it likes to call it's innerloop, but it's slow and prone to problems.  Others say that Lorito will eliminate this problem while I am less optimistic.  Lorito may alleviate the problem, but there is a lot of magic needed to eliminate it.  So, I save the solution for another day.

I wondered, can Lorito exist without goto or call?  I stood and thought, and decided that no, it can't.  Not unless we make a huge, flat map of all the instructions that Lorito knows about at any one time.  So I kept both types of operations and made the available code available to Lorito self contained.  I like things that are self contained, it's easier to handle and simpler to work with.  So I made the sheets of instructions individual for each method.

But I worried, I needed to justify my operations to anyone that uses Lorito.  So I wrote instructions on how to use each of my 44 operations and justified the existence of each.

As I looked over what I had accomplished, myself asked me a question.  What if all this is not what others what?  That's okay, I assure myself.  It's not the destination, it's the journey.  I understand the decisions I made are a departure from what Parrot is today, but this is about how I would create Lorito to support Parrot.  I believe earnestly that anything Parrot can do, Lorito can do and then some thanks to the flexibility and simplicity of the decisions made.  Its future is uncertain, but I will continue to work on it until there is no need for it, either by others or myself.

Less Magic == More Magic

(Yea, probably a bad idea.  But fun, I must admit)

No comments:

Post a Comment