State of the Indian Economy

This is about the state of Indian economy, not so much about global. October of 2019 has passed with a barrage of content everyday talking about the bad health of the Indian economy. Presumably it means that the growth of the Indian economy is stalling. May not be stalling completely but definitely not growing as fast as they expect it should. So yes, the GDP numbers confirm the story. We know by looking at the newspapers. Not sure if we can tell that anecdotally, do we see scores of unemployed people on the streets or posh restaurants and shops deserted now that wasn't t he same earlier ? Not really. I keep seeing scores of unemployed people on the streets every day and I have been seeing them even when the economy was supposed to be going great guns. And as far as expensive shopping and entertainment is concerned, those Indians seem to have a lot of money to burn, if you listened to the firecrackers this diwali or in other festivals. The restaurants are occupied enough to make any reduction in food bills unlikely. So where is the slowdown evident ? The newspapers and other media aren't lying though ! Now, after the elections, its even in the government numbers. Just 5 % it said for the last quarter. Way less than comfortable for us in India, When more than 5% is achieved by China, whose economy is more than 3 times the size of India's. Even Uncle Sam gets to 3% frequently. Why are we Indians going so slow ?

I know many people who just blame this government. They have complains about demonetization, GST and the fact that the government has struck down on black money transactions. Blast them. It's remarkable that so many people believe that its impossible to run the country without using unaccounted wealth. The complainypants even have a problem with Jan Dhan accounts because they say the rural/illiterate folk don't deserve banking services and they won't use them properly anyway. Who cares about them? Who cares about teaching them to operate banking services ? Mind you, those having a grudge on Jan Dhan accounts are actually employers who are saying that they are better off with their workers receiving cash from them as salary instead of depositing money into employees' bank accounts. You know why, right ? Yes, demonetization hurt those same people too. GST is making it slightly more difficult to hide tax payables. So here we go - those crooks are now out to complain about the slowdown in the economy. Yes, the Indian economic system is stunned into inaction, it has gotten a serious shock of a different kind of electricity, named honesty.

There is now an abysmally low amount of trust left in the Indian financial system. Nobody trusts the company books and nobody trusts the company auditors. Companies with AAA credit ratings are going bust in a few weeks, so nobody trusts the rating agencies. Nobody trusts the NBFC when it spells out its NPA's, thinking that there must be a lot more. Nobody trusts the banks either with their NPA numbers. The situation was the same earlier as much as I know - when I used to mention fundamental analysis of stocks, the wise folks used to say that *all* companies cook their books so fundamental analysis is a fundamentally hopeless adventure. Repaying loans is just not a culture here. Now that the banks are getting serious about loan recovery whenever they lend, the rich dudes just don't want to invest anymore ! Even with RERA, the real estate industry still resembles a crime scene. The lobbyists are out to dilute RERA clauses at individual state level.
Now, where do we go from here ? Who is going to invest in this economy ? So far, the deep pocketed crooks used to game the system, taxation had so many holes that for the rich, stealing tax was a given. The rich knew that they could get away with sub-standard delivery of goods/infrastructure and mint money from the government contracts. Recovering money for a finished task from the government or any private entity is incredibly tough in India. The rich now balk at investing in India because the screws seem to be tightening for them, as far as taxation and loan repayment is concerned. The foreign investors are coming to their senses about the reality in India now too.

Why is Indian labour so unproductive ? Facebook and other social media are a good explanation for employees in IT industry, I know. But simple work ethic is poor among all classes of workers here. The chinese have 9-9-6 (12 hours of work, 6 days a week), they have developed their own manufacturing industry through so much hard work, relentless sacrifices and ingenuity. Where do I see such parallel in India ? Indian manufacturing taking off is a pipe dream - Indian workers are simply not working as hard and as efficiently as what is required to make manufacturing competitive. Land prices in India are sky high, wherever you go. Wherever you go in India, we see population. Tiny villages with barely any electricity look like towns, towns look like cities with non-existent dust roads and cities look like a never ending slum expanse, of limitless illegal construction.

Talk about competitiveness of Indian industry now. Indian traditional goods involve manual labour and are works of art. Mass produced goods are made so inefficiently that export competitiveness is poor. See how the local industry captains are howling in terror when the government announced a possibility of joining RCEP ? The Amul top management is terrified that exports from New Zealand will overpower the sales of dairy in India ! Do we wonder how dairy products from New Zealand, manufactured with developed country costs, then exported 10k miles and brought to India with added shipping costs and import duties still manage to beat Amul dairy products in quality or even price ? What the hell is going on ? Amul says that farmers in New Zealand are more mechanized and run their work like an established industry while the Indian farmers are running it like a family cooperative at a smaller scale, so we can't compete. What a pity. When the government announced a deadline for gasoline vehicles and introduction of electric vehicles, the auto industry captains were up in arms. They could not imagine getting to produce an electric scooter or a car even in five years time. Bajaj and TVS keep pointing to negligible infrastructure for charging EVs. But I don't see how they act so incapable. Tesla has been running the EV show for a decade now in the US. Why can the local auto industry come to manufacture EVs in five more years time ? To be fair, the government has backed off from setting a deadline in stone and Bajaj seems to have announced an electric scooter named Chetak recently. Who knows how much they have planned on producing though. The Maruti boss is crying hoarse, wanting GST reduction and going about shutting plants and laying off workers in panic mode. Maruti seems to have 40 thousand crores in cash reserve. The auto sales slowdown is barely 20% over the last year. Why can't Maruti manage to use its reserves to support its distributors and retails in such times ? Is a 20% drop in sales life threatening to some company as established as Maruti is and that has been running stellar profits for last two decades ? How are these companies being managed ? The industry captains cheered the recent steep corporate tax rate cut to match global levels. I am afraid the industry has just got a short term let-off the hook. Now the companies can simply project an inflated profit margin for free, no incentive to gain efficiencies anymore ?

The 5 trillion dollar economy target is unraveling fast, looking like a pipe dream now. I don't know what the government has in mind for now but at least it is responsive to industry concerns and is coming up with as many tax cuts and minor adjustments. But no major reforms, no land acquisition smoothening, no labour reforms, no judicial reforms that are truly epochal. It is still damn difficult to run even a proper legit business in India. Is this government capable of taking any so called "courageous" decisions any more, even with the brute parliamentary majority that it enjoys ? They seem to be in election mode continuously. Even state/municipal elections seem to impact national level policy announcements, mostly resulting in delays in getting real reforms done. Is it really possible that a country in this twenty first century, which is still on the edge over a Mandir-Masjid construction ever get its priorities sorted to begin the monumental nation building work that becoming a 5 trillion economy is going to require ? The Supreme court of this land has spent years deliberating on the existence of a mythological figure after all.

It is going to take time, for us Indians to get out of the jugaad mentality and take a honest look at ourselves in the mirror. To stop taking short-cuts. To be diligent. We need a complete moral, ethical and cultural change.

Greta Thunberg - Crusader daring

Greta Thunberg, is the name on everyone's lips and her speech all over the news. Its truly remarkable that a sixteen year old has so much awareness of industrial pollution and concern about the environment and our collective future. Very few sixteen year olds can match that up indeed ! I hope her parents or other adults are not pushing her into this cause even though its such a Nobel noble cause.

But I feel her anger is mis-directed. Listening to her, doesn't it lead you into thinking that "If only the world leaders pay heed, we would all be saved" ? Is it just the obstinacy of presidents and prime ministers which is getting us into trouble ? Clearly, getting those leaders to sign on some document won't revert this deadly threat of environment destruction. And to solve this problem, a government dictat is definitely not the right way to go.

Greta is making an fervent, emotional appeal, alright. But governments make a move based on logical, scientific and popular appeals, not emotional ones. Industries make a move based on scientific and economic data, not emotional outbursts. So Greta's thoughts should really be re-directed to those who really need to listen - the people.

The unthinking people, really. Those jet-setting all over the world, the blind consumerist who generates enormous mounds of plastic waste, who can't stay indoors without air-conditioning, nor move outdoors without gas guzzling SUV's. Those who order Californian apples and almonds into India and then travel all the way to California to eat Idli-sambar in Indian restaurants ! When are they planning to reform ?

Elon Musk BY Ashlee Vance

The extraordinary Elon Musk.

The world knows a lot more about the Zuckerbergs and Jobs' than Elon Musk. That is a problem and Ashlee Vance eliminates it with this gripping biography. You won't turn a page without an exclamation saying "That was crazy, Elon !".

Childhood

He had exceptional lineage and a well travelled, highly educated family that valued adventure and creativity. He did not have a supportive environment at his home and yet he excelled in math and science. Even as a child of 10, he had a strong sense of purpose and deemed study of some school subjects such as the Afrikaans language as pointless, partly because he did not fathom any future staying in his home country of South Africa. But when he realized that not learning them well enough would restrict him moving to the next grade, he could turn things around and do very well in all such subjects. He was bullied in school quite badly, enough for him to end up at the doctor with blood all over his face. Then very strangely he holds up that bullying he faced as a positive moulding effect on his personality and says that his own kids who would never face such cruelty would miss out on that. Being a smart parent even as a super busy entrepreneur, he restricts his kids' cartoon watching and playing games that are not something not as soft as pressing buttons for making cute sounds and encourages problem solving. At that age, Musk would often go into his "zone" - lost in his thoughts and completely unaffected by anything else happening around him. Was it one of the most over-diagnosed Silicon valley disease - Autism ? Not likely and even if it was, it would so incredibly high functioning that it should render everybody else as diseased.

Early startups

His early startups, including Paypal were true successes of his innate adventurous spirit and academic skills. They fully leveraged his natural strengths. He caught on the ideas of a fintech company and digital payments system before anyone did and while everyone else was still making a 'unprofitable-cutesy-site.com'. This is also the time when Musk habituated into a 100 hours work week. Musk was always adventurous and risk taking and such qualities truly came into common instance from hereon.

SpaceX

The work culture in the Silicon valley and other software industries is well known and well criticised. Impossible sounding deadlines are set, employees are stressed and overworked. Never mind the what the deadlines are, they are not met anyway and there is a mad scramble to deliver something, somehow. It encourages short term fixes to basic problems encountered during testing phases. Ugly remnants of past mistakes are left behind in the product and the software engineers even have a name for it - technical debt.
Now, can any company in the business of aerospace technology operate in that kind of work culture ? Mistakes in the end product are prohibitively costly. A single snag costing a human life means curtains drawn on the company itself.
Well, Yes ! Musk and his team operated in that high octane environment for years together, could not only extract the most out of his employees in terms of effort but also in terms of innovative output from them. A Boeing or some other established aerospace player tends to indulge in much paperwork with apparently slow and wasteful procedures to make a simple device or change something that failed. SpaceX employees usually made things in-house with a tenth of the budget and time. Nevertheless, the magnitude of the requirement was such that SpaceX pushed Musk to the limits and nearly bankrupt down to the last 100 grands. Don't forget that he started with around 100M out of his early ventures.
There are several innovations that emerged out of SpaceX and some of them sound quite fundamental and impactful. Not so long ago, none of the engineers and scientists in the business of making rockets reckoned that re-usable rockets was easy or even possible. The stress and thrust of a launch was too much for any material to withstand. Musk made re-usable rockets his key to achieving economies of rocket launching and thus making his company a profit out of it. Welding together large sheets of metal could be done in a much better way with friction stress that yielded lighter artifacts.
Most remarkable is Musk's ability to get down to fundamental physics with someone while reasoning and arguing about what can be done and what can't be done. Some employees find him aggressive, brusque and demanding, so be it. He has a vision and he is shooting for it.

Electric cars

On developing electric cars, Musk continued with the Silicon Valley style of project management and work culture, with apparently disastrous consequences. The project had massive cost overrun and delay and would not have been a viable show unless Musk pumped his personal wealth into keeping it going. One must say though that the if there was anyone who could build an Electric car that captured the imagination of ordinary folks and inspired the rich to buy a sports electric car, it had to be Musk. Musk did'nt want to just build a higher range electric car, he also wanted the car to go from 0 to 60 in 4 seconds. Anyone else would never have begun something so ambitious. Musk is a visionary too, and the first one to realize that improvements in the Lithium-ion battery were sufficiently good to make a full range electric car possible. Conventional car companies of the time and even today will be hard pressed to identify innovative ideas of the calibre that Musk came up with, such as free charging stations for his electric cars, touch screen controls and automatically extending door handles.

Overall ...

Musk has set for himself the loftiest goals, that most people would consider borderline insane. He wants to save humanity by accomplishing the staggeringly ambitious feat of interplanetary colonization, particularly on Mars. Note that while planning to save humanity, he didn't think of making efficient seawater de-salination or controlling global warming or making vaccines or building sanitation for all. He goes all flat out for the fantastic. And continues to calculating the number of rocket trips it might take to send people to Mars. And he could explain all this to someone while finishing dinner in a cafe with a dollop of desert sticking on his chin.
But in aiming that high, he has already accomplished so much that was previously thought impossible. It might have taken more time and more money than he imagined but he has clearly done what he declared as a near term goal !

Does this world need more Zuckerbergs and Bezos' and Jobs' ? yes.
But do we need more Elon Musks ? Bloody sure yes !

Book Review: How Fund Managers are making you rich - by Pravin Palande

If you have been investing or even thinking of investing in mutual funds in India, you should be really curious of how the folks supposed to handle your hard earned money work. Their background, habits, methods, where and what have they studied or smoked ?
Finally, here is a book with a slightly presumptuous title clarifies most of those things.

Before moving to individual fund managers, Palande provides a good overview of the Indian capital markets. It starts with talking about the period that most of us don't remember fondly - the dot-com bust of 2000. He quotes prominent fund managers of those times who saved their fund value by not falling for the hype and madness surrounding the IT stocks then. Its all about the steadfast fund managers whorefuse to invest in the fluff companies whose business they don't yet understand while the markets were on steroids due to unreal valuations to software companies. They get the brickbats when the fund returns lag the benchmarks and how the smiles were back after the crash.

Palande also brings out the small and big innovations that have come out of Indian fund managers, such as the low cost Benchmark mutual fund ETFs and distributor-less Quantum equity fund.

In the second half of the book, each chapter is devoted to an individual fund manager. It goes from how the manager started his career, moved between fund houses, evolved their own thinking about the markets, their predilections and their methodologies. Each one is a fundamentalist - or someone doing a fundamental analysis of the stocks but each one still has some uniqueness. So much that the chapter can be tagged by their most noticeable idiosyncrasy. Each chapter gives us a view of the nature of the fund managers job and
their uniform assertion of holding humble respect towards the unpredictable beast that is the stock market.

I would have liked some more description about the software systems that the managers use and the kind of data they analyze in the book. Some details about how they decide how much cash ratio to maintain and stock allocation. Do the fund managers really have some secret weapon at their disposal that puts them at a distinct advantage of the ordinary retail investor (apart from their superior knowledge and sense of the stock market, of course) ? They do have a research team and can afford a full-time attention to detail. Still is there anything else ?

Palande ends the book with a chapter that is bound to raise the hackles of the mutual fund gurus in India. He makes a case for investing in low cost Index funds. A majority of the mutual fund managers have similar kind of data gathering and analysis mechanisms at their disposal. Palande reasons that due to a very secular availability of information and maturing of the Indian capital markets, there is a progressively low variance in the returns that mutual funds provide. Indeed, it is getting very difficult for mutual fund managers to make their funds return a premium over each other or even the index. Further, the trend is set to harden, so why put your money in an active fund that charges 2.5% + commissions versus an index fund that does with less than 1% ? The Indian mutual fund industry counters by saying that such a case for index funds is already ripe in American markets but the Indian mutual fund manager can still beat the index that justifies the higher expense ratio comfortably. This is probably the point where the CEO of Birla Sun Life AMC does not agree with Palande, that he mentions in the foreword.

So please go ahead with choosing quality funds and fund managers for now... Yes, but they only can make you rich if your are into the mutual funds or the stock markets !

More code optimization notes

Continuing from my earlier post about code optimization techniques here.

This is all about making the code more data oriented. Which means that the code is written such that its data is organized intelligently in memory, considering CPU cache access patterns and data sizes.

As it is, extracting benefits out of any of these techniques is a black art. Some may back-fire in specific circumstances. So, benchmark... be absolutely sure to measure(twice) before cutting(once).
Time to consider a few more ways to make code run faster...

1. Inlining functions: Yes, nothing too special about this but worth noting. Take care about its impact on size of the executables.
   
2. Mark functions as constexpr: The newer C++ standards make it simple to go further than inlines. If you tell the compiler that the function can compute statically for all its inputs, make it constexpr so that the computations finish at compilation stage itself. Of course, the inputs that the callers pass to such a function should also be known fully at compile time. Inlining the function produces no guarantee of actual inlining by the compiler but if we do make the function as constexpr, we are sure that there are absolutely no runtime costs of this function at all.
   
3. Provide specialized versions of a function per data type. This is not just about your function taking a polymorphic base type as an argument. C++ allows overloaded function names. It is sometimes possible to write a function slightly differently for a particular data type so that it is more optimal. Perhaps it saves you a remote call or two. But at a very minimum, if it helps runtime conditional checks, it could be worth doing.
   
4. Unrolling loops: Doing more actions in one iteration of a loop.
   

   for(int i = 0; i < 4*x; ++i){
               // process(arr[i])
             }
       

   changes to...

   for(int i = 0; i < x; i += 4){
                  process(arr[i]);
                  process(arr[i + 1]);
                  process(arr[i + 2]);
                  process(arr[i + 3]);
             }
       

   which just saves some book-keeping conditional checks and increments in the for-loop that were happening every iteration.
5. Pass values for anti-aliasing: Sometimes you have got to assure the compiler that some value is certainly not going to change for every iteration of some big loop. Then the compiler will not add a fresh load for that variable.
   

   void func(int arr[N], int* p){ // 'p' passed as pointer. Not sure if it is actually aliasing some arr[i].
                  for(int i = 0; i < N; ++i){
                      update(arr[i]);
                      // also uses 'p' 
                     // load 'p' again to ensure it is latest.
                  }
             }
       

   Here, we could pass p by value, that is good enough for the compiler.
6. Avoid guessing for branch predictions: Sometimes, calculating all conditional paths in a piece of code and then deciding at the very end what to return could help save from a bad branch prediction cost.
   
7. Process large arrays in parallel. Batch them and take care to avoid false sharing, i.e - threads that still access the same data and then block on mutual access. Make sure each thread gets a full cache line.
   
8. If you process something in parallel as in the previous point, you might have to reduce at the end to get a combined result. Reduce pair-wise instead of making all threads block on a single shared data.
   
9. Unroll loops based on latency of inner operations and order the operations so that maximum registers/cache is used at each point within the iteration.
   
10. Separate fast and slow code. Remove exception handling out of tight code.
    

That's it for now.
Hoping to be mindful of such finer points.

Find the permutation !

I first thought that solving this problem would require much longer piece of code ! Urge you to have a go at this first.

The first thought is to see if each inversion sequence has a one-to-one correspondence with the actual permutation. Given a permutation, its inversion sequence is obviously unique. Other way, if an inversion sequence has two different permutations, then we can start verifying numbers and their positions in ascending order as follows - Suppose x is the lowest number whose position does not match in the two permutations but their inversion sequence is the same. But the higher of the inversion values is not consistent since it must be made of the numbers lower than itself. So the permutation is unique too.

The brutest of brute force ideas for this problem could be to just generate all permutations of the input length and see if the criteria is met.

So the clear next approach is to try generating the permutation by considering the positions in the reverse order. Since the inversion sequence (input) only worries about numbers lesser than the current appearing to the right, we use the reverse ordering while writing out the output permutation. So for the permutation of numbers from 1-8, we should be able to write out the answer by deciding the position of '8' first then descending towards '1'.

At each step, we keep track of how many numbers lie to the right of the current position. And here is the core problem - For a permutation of size 'N' numbers ('N' in 10^^5 range), how to maintain efficiently the relative position-from-the-right for each number while some positions keep getting filled iteratively ? O(N) numbers and O(N) per number meaning overall O(N^^2) is not an option with that large 'N'.

The next option is maintaining an augmented balanced R-B Tree wherein each node keeps the count of nodes under it. Too long and complex to code...

Another data structure that comes to mind is a Binary Indexed Tree - BIT . This will help fill up a position and then update the relative positions appropriately in O(N.log N) time. This would require referring to some known implementations.

Somehow I felt that this should require something less than a BIT in full generality. Basically I got this -
1. Start with an array of size 'N' filled with values equal to the initial relative position from the right. So the values are in descending order from '(N - 1)' towards '1'.
2. Going iteratively in reverse order as thought earlier, do a modified binary search for the right position.
3. During the binary search, whenever going right, mark the current node to say that its relative position will reduce by '1' since we are on the way to fill up a position towards it's right.
4. When traversing to the left, increment the position that is sought since we know that those many positions to the right have been occupied !
5. When the correct position is found (sought position matches the calculated position), mark that position as deleted but nothing to delete from the array or shift values.

This takes O(log N) time for each of the number being placed with an overall O(N.log N)...

Here is the "accepted" code I got,

#include <iostream>
#include <vector>

class Position{
  public:
    int pos;
    int change;
    bool deleted;
    Position():change(0), deleted(false){
    }
};

int bin_search(std::vector<position>& vecpos, int seek, int left, int right){
    if(right <= left){
        vecpos[left].deleted = true;
        ++vecpos[left].change;
        return left;
    }
    int mid = left + (right - left) / 2;
    int curr = vecpos[mid].pos - vecpos[mid].change;
    if(vecpos[mid].deleted == false && seek == curr){
        vecpos[mid].deleted = true;
        ++vecpos[mid].change;
        return mid;
    }
    if(seek > curr){
        return bin_search(vecpos, seek + vecpos[mid].change, left, mid - 1);
    }
    else{
        ++vecpos[mid].change;
        return bin_search(vecpos, seek, mid + 1, right);
    }
}

int main(){
    int N;
    std::cin >> N;

    std::vector<int> inv(N);
    std::vector<position> vecpos(N);
    std::vector<int> ans(N);
    for(int i = 0; i < N; ++i){
        std::cin >> inv[i];
        vecpos[i].pos = N - i - 1;
    }

    for(int i = N - 1; i >= 0; --i){
        int pos = bin_search(vecpos, inv[i], 0, N - 1);
        ans[pos] = i + 1;
    }

    for(int i = 0; i < N; ++i){
        std::cout << ans[i];
        if(i < N - 1)
                std::cout << " ";
    }
    std::cout << "\n";
    return 0;
}

This works because for any 'leaf' node position, we always use the same binary search traversal !

It's always interesting to see patterns linking linear structures like arrays with hierarchical ones like trees.
The connection almost always requires something novel.

Comments on : Functional programming in java script

I highly recommend a wonderful set of interactive exercises here !

It starts with set of exercises for functional programming but later on takes an unexpected turn and goes on to demonstrate features of Microsoft's reactive extensions library for java script. Nevertheless, I jumped into it initially being curious about functional programming techniques common to java script.

Starting with functional programming, there are at least 25 exercises that show how defining a small set of simple abstractions can lead to short and powerful programs. I could work out defining and using the five primary functions (map, reduce, filter, zip and concatAll).

Totally convincing exposition of the power of functional programming - the way that truly complex business logic can be described by surprisingly few lines of code involving just intelligent application of those 5 primary functions ! Basically this was the most 'fun' part of this tutorial.

Later on, not knowing how asynchronous coding and the functional programming techniques are related, the reactive extensions library is introduced. It was striking to learn and find similarities between traversing arrays (using the functional techniques) with processing events (using special objects called 'Observables'). Observables are supposed to be (possibly infinite length) sequences of events. The sequences relate to arrays and thus we can use functional programming primitives to process them. Being observables, though, allows the object and underlying sequence to change over time. Whenever the observable changes, the listeners can run code to process the event.

So the crux of the tutorial is to show how the same kind of programming techniques can help process any kind of data collections or events. It is a unified way of dealing with synchronous data like arrays as well as asynchronous objects.

I am quite used to multi-threaded programming with C++ but not exactly with writing asynchronous java script code that is commonly involved with event handling. So even though I have not really faced the difficulties in handling asynchronous actions in code first hand, I can still appreciate that asynchronicity is bound to present considerable complexity in writing efficient and correct code. The tutorial also comes with examples that show how complex the code can get once certain asynchronous calls are made in parallel and we usually define success and failure callbacks on each request. If it is required that some activity happens once *all* parallel tasks complete then then we have to check and track each request status and perform accordingly.

I still found the narrative promising - that the unified programming model is very helpful in managing that complexity. Functional programming strikes one as hard, sometimes I wondered why one should bother about functional stuff when the equivalent procedural code was so easy and a mechanical activity to write (even if it was 20 times the size). One has to get over a significant learning curve to understand why.

But that is how programming has evolved it seems over the years, in pursuing procedural code for a lower learning curve, we now realize that some things cannot be too simple.