Monthly Archives: January 2009

Living in the Glass House

The modern business is highly confidential.  Companies have the deals which never become public, employee may have special conditions which no one knows about too.

This is just life and if you do not respect confidentiality you will have hard time finding people to do business with you or work for you.  But what you can do is to treat your deals as public.  What if your partners/employees/customers would know about the deal ?  Would you be comfortable with it ?    If you’re not the good chances are the deal is unfair and you should not get into it.     I also try to apply the same principle to employees – OK their salaries are confidential but what if they would all know how much everybody earns and how the compensation is structured ?  I would like to not to feel good about it.

Of course deal fairness and salaries is sensitive topics and if indeed everyone would know everbody’s terms and conditions some people would not be happy.  Everyone has a different judgement – employees may have a different feeling about their performance compared to someone else and also customers and partners may have a similar feelings.   Keeping everybody happy is impossible but making sure you feel good and fair about any deal is.

What if houses were cars ?

What if people would treat houses same way as we treat cars – meaning you purchase the car, often taking a loan for a few years.   Quite similar to the morgage,  just different loan amount and length.    The difference comes from the way we treat these loans –  with a car we know the value will go down and the car will become close to worthless when the loan is fully paid.  With houses we however assume price will just go up.

Things would be much safer if people would threat houses just the same way –  purchasing houses to leave in it,  considering morgage and expense and making sure rent covers the morgage and related expenses if this is house for rent.    Increase of house in value should be seen as an extra bonus not granted.

If people would treat the houses this way  the drop in houses prices would be rather seen in a positive way.    If you can but more   stuff for your dollar, stuff being cars  gas or house  is good for consumer.  Deflation can be concern for global economy but this is different story.

Salaries in a Virtual Company

One big challenge in a Virtual Company with people working from all around the world is setting the Salaries.

One approach which MySQL used when I worked there and which Zak and Monty propose in their Open Companies idea is to base salaries on the local market where employee lives.

I never liked this approach myself, in particular because I was for long time on the “suffering side” – working for MySQL from Russia and paid fraction of what people doing exactly same job, serving the same customers were paid in Europe and USA.     I think this thinking is the remainer of “office culture” for international companies when different  subsiduaries in different countries have their own salary guidelines.

This approach also poses other interesting challenges.  For example I knew about number of people in MySQL who would like to immigrate to the country with better quality of life but MySQL would not support them because this meant paying more without getting any more value.   I also know cases of the opposite kind with people moving back to the countries with lower cost of life… and having their high salaries kept (there seems to be the big no-no about reducing salaries in the Corporate America) which makes it even more unfair compared to the guys which just did not move anywhere.     I should mention though, I was an exception myself with being able to move to US while working for MySQL and get an appropriate raise, though I also arguably could bring more value while being in US – being able to do onsite consulting etc.

As you might quess at Percona we have the same challenge, though we approach it differently.   We define fixed budget and position requirements and look for people around the world.   For some positions, for example dealing with customer accounts perfect spoken english and working in US business hours is actually requirement so we have to hire US guys and pay salary by US standards.     For a lot of other things, like internal systems development there are no location specific requirements and so we can keep our budget for these positions uniformly low, which makes people mostly from eastern europe to apply.  In fact we do not even advertise these in US as these would just look funny.      Finally for Consultants  (the ones who really do all the customer work) we use revenue sharing system which is the same independently from where they are located.  True there is some benefit being in US – being able to work in a business hours for US customers and being able to do onsite consulting  but there is also the cost of higher living expenses.   We also  try to focus on the “position cost” which due to different tax structure in different countries and state may significantly differ from what “gross salary” is.   This goes in line with exactly same thinking –  for virtual job (which can be done from whatever location in the world) it is final cost which matters.

I also like to describe it the following way  – the world is a big town these days, especially for virtual company ?  Will you pay higher salary to someone who likes to have a posh house  in expensive place compared to someone who choses to have long commutes to keep his expenses low ?  I do not think so.    And what we have is just applying the same model on the global scale –   as long as you can make to the work in time and do work of the same quality we do not care where you live and will not pay you differently.  You can move arounds the country or around the world as you wish  but do not expect this to affect your pay.

I think taking this approach eliminates a lot of conflicts and give a lot of freedom to our employees.

One hard related question is currency rates.  These go up and down making same salaries nominated in the local currency, considerably different over time.      Our current solution was to nominate salaries in US dollars, which also matches the fact most of our contracts as billed in US$.    As we employ people around the world (so far everyone outside of US are contractors, even though we treat everyone as employees) we will have to come up with some other solution. May be re-evaluate salary monthly to be ballanced ?   This may not sound fun (because it means salary can do down as well as up) but I think this is what we just need to accept while working on the global scale.

On Dell R900

We recently got Dell R900 box to use as a benchmarks so in addition to looking remotely to a lot of them we could pick an eye inside.

Inside it is nicely designed box with everything pluggable.   It has 32 memory slots making it possible to get 128GB of memory with less expensive 4GB DIMMs.   It takes 4 quad core CPUs and comes by default with 4 network interfaces….  It really looks like it is designed to be  virtualized as 4 boxes in one.    Indeed you can get 4 virtual slices with 4 cores each, dedicated network card and 32GB of memory.  It also has plenty  of PCI-E slots for extensibility.

What I did not like though (and this is my general concern with Dell/HP/IBM/Sun) – They have very low limit on number of external drives.    This is 4U box which is  twice as high as PowerEdge 2950 but it still only takes 8 2.5 inch drives.   The rest of space is used just for the fan holes.

I do not believe this is done for cooling reasons, especially as SuperMicro manages to fit 24 of such hard drive slots just 2U chassis.  I think it is just a silly trick to make people buy external storage if they need more drives, which is typically the product with higher premium.

Though it is a good question if we will need so many drives in the future.  Most people getting many drives these days do it for performance reasons and SSD may eliminate this need… and even 8 SSDs should be offering enough space and performance for many needs in 1-2 years timeframe.

Winter in Bay Area

There is no winter in Bay Area….  no Winter as in Russia with its snow or Winter with it rain and bare trees as in Seattle or London.    Autumn seems to change just to the spring here.     It is the middle of January but you can still see some trees which still have some last green leafs remaining, while at the same time there is variety of new growth.

I like it especially as the real “snow winter” is easily reachable within 3 hour drive in Tahoe or Yosimite area.

Are all rates created equal ?

When we look for professional services, being it your plumber accountant or lawyer  we pay special attention to the rates,  and as vendors know it they often would focus on making rates to look low while the fineprint results in your bill being higher than expected.

For example when I was moving and hired moving company to help me with the move  I mainly looked at the rates while in the end there were a lot of cheating and a fine print.  For example there was also fuel surcharge, driving time counted double.  Some items MUST be packed to be moved, like  moving my TV would cost $150 even though I picked it up at the garage sale for half of that price so I would rather get couple of extra scratches.  When there were extremely high usages for supply –  something like $10 per table roll and they there very generous to themselves in using it when it comes.

This is not to mention general performance questions – the movers were students spending few months in US on Work and Travel program so they were not overly experience and performant moving the furniture, and in the end we got scratches both on the furniture and on the walls in addition to getting more hours than I would anticipate.

I was surprised with larger than expected bill and surely will never hire the same company again but I learned a lot about picking moving companies and also get an extra assurance we made the right choice with Percona rates – we keep them high enough to be honest and well performing.

In fact we go much further than a lot of people.  Some consulting companies I talk to say  hiring our consultant is same as hiring employee – will you pay your employee if he meets the new challenge and have to do some reading and digging ? So you should do for consultant.  Not with Percona.   We make sure general education time is not billed to the customer,  learning how the system works – yes but not underlying technologies which we claim to be experts with.

What if consultant have made a mistake ?     At Percona we make sure not to bill for mistakes or other inefficient work as well as for any time required to fix mistake.   This is how I would prefer to be billed when hiring professional services and I think this is good for consultant too, to learn to be careful.

What is about individual performance ?  In particular in even very smart group there are going to be people more junior than others, or less experience with particular task.  We think customer should not suffer depending on this factor and so we make performance adjustments as needed so only the time it would take the the most experience person to do such task.   This is where we learn from Auto Service industry when there is a set amount of hours for the task no matter how long the job really takes  – however because all our jobs are so different we can’t simply use fixed amount of hours say for “query optimization”, but the idea is different,  with exception there is no downside for the client –  we never will bill 1 hour if in this individual circumstances job took just 15 minutes.

Finally comes question of efficiency.  This is the big challenge because natural rate structure of consultant being paid by the hour means he is motivated to spend more hours on the job.  We believe our higher rates do us a favour here too because we know customers look closely at out timesheets and will question if they consider time is wasted.   But really it almost never comes to that  – we make sure our consultant focus being efficient and doing what client really needs to be done rather than wasting hours, plus we have the process of internal time sheet review to ensure customer really needed the work done and it was done with a good efficiency.

This is only one side of efficiency though – the other is how long it takes to do the job ?  We think it is about people, teamwork, processes and tools.  We hire the best people, our consultants work as a team picking into each other experiences as required, we ensure to have efficient processes for doing things, for teamwork facilitation and training and we actively invest in tools to automate our work.   The maatkit is the most well known one though we have done various other tools and MySQL patches

In general, with all these things to make sure client gets a lot for a billable hours I think we’re in as much as 2x efficient in terms of billable hours compared to traditional consulting company and more than that if you compare to inhouse employee, though I do not have a hard data handly.

Is it a right strategy to have  more expensive but more valuable hours ?    I think it takes a little customer education and overcoming bean counters resistance which assume all rates apply to replaceable hours but it is good for everyone in the end.