What do mathematicians and statisticians just "get" that other people don't?

Along many flaky-spooky-whatever dimensions humans can be ranked, I would highlight one: the level on which one thinks about the future.

There loosely are three levels of reasoning about the world with respect to one’s actions:

⒈I have no idea what is about to happen.
⒉I understand some basic if-else rules.
⒊I am aware of what the probability distribution is, I view the world as some Markov-ish chain of what’s going on, and the way my brain operates can loosely be mapped to traversing the graph of this Markov process.

For a very simple analogy, (1) would be someone who medically qualifies for a moron, (2) would be able to learn tic-tac-toe or chess or Go, (3) would be able to excel at poker or to be an entrepreneur.

(Note that what I am talking about here is not the cognitive capacity of an individual, but whether they have internalized certain process of perceiving the world rationally in their mundane activities.)

What most reasonable people get to is understanding the “highly likely / highly unlikely” events. For example:

∙ This job is way over your head. If you go and interview, it’d be a miracle if you get it.
∙ This person drives a lot while being drunk. It’s most certain that they are going to get in trouble with that. No way I am getting into their car.
∙ Those guys are going out together for a while, and both have their leases expiring soon. There’s a decent chance they are moving in together starting this Fall.
∙ This candidate talks too much nonsense, there’s no way they could win the elections.

For judgements on life situations like the above, the human brain is generally capable of categorizing them into the “highly likely” / highly unlikely” buckets. If something doesn’t go our way, we get somewhat surprised, but since it does happen from time to time, it doesn’t break our mental model of the world.

The above is how most people appear to live their lives. What mathematicians and statisticians get is that there is a tangible, noticeable difference between something having an 0.01% chance of happening, 0.1% chance of happening, and 1+% chance of happening, and we are able to incorporate these subtle differences into our decision making processes.

For an obvious example, imagine you can have something done in five minutes with a 99% chance success rate, but in that one percent chance it doesn’t work out you’d have to spend half a day cleaning the mess.

If it is one's work five days a week for eight hours, there’s a good chance they will converge to either “I’m doing simple things one after another, and sometimes I have to clean the mess", or “my job seems to be all about cleaning the mess, because every single week something goes wrong”.

Someone who understands statistics would immediately realize that if one of a hundred times five minutes become four hours, which is 240 minutes, then of a hundred times there will be 99 * 5 = 495 minutes of cruising and 240 minutes of fixing shit. Thus, with these odds, they’d be destined to spend roughly one third of their business hours cleaning up that shit. Hence, it’s perfectly justified to invest into improving the process, because, for instance, if spending six minutes instead of five and improving the odds of success from 99% to 99.5% would make this workplace significantly more enjoyable, it should absolutely be done.

People who understand statistics view all or most life opportunities as “X risk Y reward”, and are not hesitant to aggressively optimize the shape of this curve by picking which opportunities to engage in and how to balance them.

I’m sure the above could be phrased better, but the TL;DR is that for most people the [0.01% .. 1%] and [99% .. 99.99%] probability ranges are just points close to zero and one respectively, while for people proficient in statistics they are extremely wide opportunity windows waiting to be utilized accordingly.

[1/3]

What are some examples of web applications that should be designed as monolithic instead of microservices?

Call me old-fashioned, but I’m still a big fan of monolith.

Let’s carefully examine the most cited pros of microservices:

1. Easy to develop new modules, as only a part of the existing system has to be brought up to work on the integration.
2. Shorter build / startup times.
3. Better scaling in case the service is already big, or if we have good reasons to believe it will be big soon.

The cons of microservices, to my taste, are:

1. Versions compatibility hell.
2. Data and API compatibility hell.
3. The errors of various parts not talking well to one another are hard to notice, and thus a lot more extensive testing is required.

If you ask me, the only plausible reason to favor microservices over a monolith is the (3) in the pros section above.

Simply put, if you run a service of the scale of Facebook or GMail or Netflix, chances are, microservices are a better way to go. For something smaller, down to and including even Quora, I would still vote for a monolith.

Justification:

• Well, the database itself will be in and of itself distributed. In other words, it will scale.

Depending on the choice of the very DB, and on how much work is designed to be performed by the DB engine, the layer that scales “automatically” can be thinner or thicker, but, generally, API-wise, the data access tasks are not the ones to worry about.

For example, in case of Quora, retrieving answers per question, or per author, or per topic, or even similar or related to the one you are reading now — these tasks can be more DB-centered (ex., stored procedures in Postgres), or backend-API-centric (if the DB is a thin layer, and the backend itself is a bunch of stateless, horizontally scalable pieces of code in Python/node.js/Go/Scala/etc. to perform those retrieval tasks).

• Some backend tasks are best to be isolated.

In other words, they may look much like microservices from the outside, but in reality it’s just a whole different subsystem.

Quora-level examples could be the ad platform, or the stats engine, or the e-mailing mechanisms. Well isolated, hardly talk to each other outside super-straightforward APIs, and those APIs are relatively easy to version.

Basically, if a module A develops as versions A.1, A.2. A.3, etc., and module B develops as B.1, B.2, B.3, and if it is simple for A.3 to fully support the APIs of A.2 and A.1, and for B.3 to fully support the APIs of B.2 and B.1, then splitting those services makes perfect sense to me.

Oh, and add the DB schema into the picture as well. The data in the database(s) will inevitably change, and not only with new fields added, but with the old one removed, or, even worse, changed in meaning.

• What I am coming to that at some point supporting way more than one previous version of the APIs and the DB schema becomes exponentially harder than supporting just one, current, version of the application.

And by supporting I mean everything. Developing a few features concurrently doesn’t look hard. Unit-testing it is not hard too. But then come integration testing, performance testing, maintaining / troubleshooting / digging through the logs, and many other issues with possible friction between various versions of various parts of the system.

And then comes the need to support several older systems for quite a while; sometimes forever. Payments, for example, especially the external ones, must always be there, in case of having to re-send the transfer, re-route it via a different channel, or [re-]create tax forms, for the customer or for audit purposes.

[2/3] (What are some examples of web applications that should be designed as monolithic instead of microservices?)

To summarize:

• If the components are totally independent, and can or should scale by themselves, I would support thinking of them as microservices.

A good litmus test would be whether it’s okay to develop those components in different languages, and run them on different machines.

Bonus points if they could possibly have different DBs; if you can run component A on some canary read-only copy of its prod database from yesterday, and couple it with component B that runs locally with its own, mock, database — here is a solid case for a separate service. It won’t necessarily be a “micro” service, but you get the idea.

Following the Quora example, examples [sub-]services could be: the main Q&A platform, the ads engine, the e-mailing system, then payments, stats reporting, private messages (imagine Quora launches a real chat), etc.

• If the components are substantially interconnected, so that they can depend on their respective representation of the data in the database, or even so that they may, at times, be better off calling one another’s internal APIs, I would strongly call for merging them into a single, monolithic, service.

It’s just the religion of strong typing applied on the system design scale. If some error could be caught by the compiler, on the level of statically checking the source code, why not leverage this benefit 100% of the time?

Yes, module build times will get slower. So what? In reality, it only affects very small features, which one could launch in under a day, if those five-ish development iterations would take one minute each instead of ten minutes each. And, yes, those small features will be some 2x longer to develop. So, yes, one-liner changes get slower to deploy end-to-end.

For pretty much any larger feature, especially the one spanning multiple teams and requiring several people from each of the team involved, I would say it is very beneficial in the long run to make sure that the code should build, run, pass not only the unit, but also the integration and regression tests, and, most importantly, this code should be reviewed by a member of each of the teams involved in the paradigm that assumes shared ownership of the common codebase.

And there is no better way to ensure the teams assume this shared ownership than to ensure that what they are working on is a single module, not a constellation of “micro” services randomly talking to each other.

[3/3] (What are some examples of web applications that should be designed as monolithic instead of microservices?)

As a footnote, I should add one compelling argument I often hear offline when discussing the above: the value of containerization during development.

Call me old-fashioned again, but here comes the gist of my experience so far.

Generally, in production, whether something is containerized or no is not really important. It could be many independent [virtual] machines, or a few beefy servers running isolated container, or a few beefy bare metal machines running all the services combined without any virtualization or containerization. Let the DevOps tell engineers how would they like the code shipped and run.

(Yes, DevOps would generally love to have independent containers, but that’s a whole different story.)

The point I do accept wholeheartedly is the idea that once many micro-dependencies are containerized, the developer of a new feature would have zero problems spinning up just the right versions of those sub-components. Thus, if, for example, those various versions need different versions of certain node_modules, or a different version of the whole node altogether, it would not be a problem for the developer at all.

Yes, from this standpoint, consciously developing a large application as a bunch of microservices is a strategy that makes sense.

Personally, I would argue that this approach is:

• not a solution to the problem, but just a way to postpone it until major issues hit, and

• only valuable when most engineers on the team(s) are not senior enough, but enabling them to build and ship various features quickly is more important than ensuring the overall quality of the product.

However, even taking these two points into account, one could easily counter that in a company X the above is precisely the case: we need junior-ish engineers to ship features relatively quickly, and, generally speaking, it’s okay to postpone most engineering problems, because most features would die out soon regardless, and the ones that stay will be (or have to be) re-implemented “the right way” sooner or later.

As an engineer, I would prefer to stay away from such an environment (that’s why I shy away from overfunded B2C Silicon-Valley-style startups), but as a business person I completely endorse the idea of moving fast and breaking things in the fields where what the company needs is exactly this: move fast and break things.

[ 1 / 2 ]

On the boundary of consciousness & AI research a lot of concepts can be declared illusory. I find it fascinating to think how much simpler does the rational logic get when we accept for a moment that the concept of understanding is just that — an illusion.

Take a complex enough task that is from the domain where the algorithm of finding a solution can be formalized as a relatively straightforward computation. Games are a good domain to illustrate the concept: In the order of increasing complexity we have some Tic-Tac-Toe, then Sudoku, then Chess, then Go, then many others.

In case of those games, the very existence of a piece of a software (plus hardware) that consistently outperforms us, humans, proves beyond reasonable doubt that certain computational problem is solved by a computer once and for all. Once such a proof exists, It is foolish to argue that a human could possibly do a better job at this game than a computer.

Imagine trying to raise a kid today with a belief that, with no external "enhancement" to their intelligence whatsoever, we can make sure that by some age twenty they would beat a computer at a game of Go. To me, as someone who understands some computer science, some statistics, and something about both education and genetics, that's an absurd goal. It likely is a lot easier to raise a kid so that by the age twenty they manage to design a drug for prolong human lifespan to some 1,000++ years than it is to raise a kid who by the age twenty would be able to be better at Chess than today's top pieces of software (plus hardware).

So, yes, while we used to believe there is some "human intuition" component to Chess (and then Go and many other problems), once the milestone of the computing machine surpassing the level of human comprehension is reached, this "intuition" concept does no longer apply. In a way, this "intuition", most certainly, is still there; the computing machine just possesses "more" of it, as evident by its performance.

What I am coming to is that to accept that one day the AI will surpass humans once and for all it only takes accepting a single trivial hypothesis: a hypothesis that "human intuition" is an artificial, illusory concept that, in and of itself, is of no special value or meaning in the Universe. Yes, naturally, human intuition is something we, humans, subjectively cherish as something that makes us unique and special; subjectively is the key word here though.

(Note also how we, humans, keep conveniently re-defining what "human intuition" is, so that there's more left to cherish, as the objective reality of constantly improving software and hardware is piece by piece eating up what we used to consider "special" and "unique to humans" just in the recent past.)

This quickly gets onto the territory of psychology and theology, but the surface is very easy to scratch. Basically, I claim that I'm ready to buy into the simple idea that, for any measurable task, a computer will sooner or later outdo an "unaugmented" human (or a committee of, possibly augmented, humans, FWIW) once and for all.

Because the idea that we, humans, just "understand" some aspect of our life better than a computer possibly could is, again, an illusion. Also, note how I carefully use the word "computer" here as a broad umbrella term for a combo of "the computing device, plus a piece of hardware equipped with the respective software". Because an abstract algorithm can not be the golden standard here, it has to be not a only thoretically possible in some future, but a practically implemented today solution for the above logic to hold. This paragraph is the hedge against arguments such as using quantum computing before we truly understand what could the spectrum of the problem they solve look like.

[ 2 / 2 ]

Now, much like a computer can already do better than humans at route planning at city scale, a computer will soon be better than humans at more and more social tasks, from medicine to how to better provide people with food and housing. And for each and every task, once the computer has proven itself effective, it's a very short time span until the humans, whose lives it just improved, would begin to happily follow those computer-generated scripts and instructions. Much like we, ordinary humans, are happy to follow the instructions of our GPS navigation systems (or the suggestions of the dating apps, FWIW), city / factory / farming / country / continent / interstellar planners would begin following, if not praising, the instructions of the respective computers who just can do better and do, consistently, do better that the best of us.

Because the very idea of "humans know better what it is for humans" is a naive self-serving belief, only good for us all to retain some optimism and mental health. Evolutionary, we have to "believe" in something, which contains a component along the lines of the continued existence and the prosperity of our species. This belief translates to various instincts, experiences, thoughts, actions, and rationalizations, and this almost-religious (if not entirely religious) belief that we, humans, have something unique to us that can not possibly be replicated and outdone by a computer, is just one of the many forms it takes.

Note that the above logic does not in any way require any beliefs about the nature of human (or any other form of) consciousness. In other words, even if there happens to be an irrefutable proof that human consciousness is unique to humans, none of the above would quit being true; and it would still be true if there's a solid proof that human consciousness is absolutely nothing special on the face of our planet or even the whole Universe.

In other words, some human's actions (or the actions of some group of humans) aiming to establish the grounds for The Humans as the masters of our planet (and/or city/country and/or the Galaxy/Universe) are probably best to be seen as one way the many evolutionary mechanisms, sharpened through natural selection, manifest themselves into certain activities. In the modern, convoluted world, instinct-driven actions can generally be rather sophisticated — as some examples, take choosing to convince your kid to get to certain school, or presenting someone special a gift of a diamond ring with a clear purpose. So, some pro-human and against-AI stance, neo-Luddism if you wish, is actually a demonstration of the force that has the same nature than patriotism or tribalism or nationalism, to name a few.

It's funny the word human-ism is being so horribly misused these days. Because it clearly should be a bad word, which totally discriminates against other forms of intelligence, that is far superior to ours. Along with many anti-ism-s programs, a program to contain the spread of human-ism is probably in order some time soon 🙂

Here's a "serious hypothetical question".

Imagine the civilization as we are today approaches a planet with a civilization as we were in some 1969.

They can already launch stuff into the orbit, but it's more scientific and exploratory; we can already defend our satellites if necessary. In other words, if we have an ISS on their orbit, there's a good chance they won't be able to take it down even if they badly wanted to.

Of all the questions in such a hypothetical scenario, one bugs me most:

Given what we know today about fake news and otherwise manipulating people by means of propaganda, would our civilization today be able to accomplish some kind of conquest-like goals by means of only spreading [dis]information, and perhaps occasional bribing of a few individuals by selectively providing them with the information we already possess thanks to our satellites, while they do not?

* * *

If yes, what would the most effective strategy be?

With dictatorships, it would easy: be a reliable source of actionable information those very countries want to hide from their citizens, and watch those regimes fall.

With excessively religious societies a solid strategy could be to reliably pose as prophets "who live in their TV/radio", until they begin to follow us, not their own leaders.

With egalitarian, self-sufficient societies I can't see a quick win, except targeting them with the neighbors of the above two kinds.

* * *

For bonus points, say there are two civilizations as "advanced" as we are today approaching that "1969 humans" planet.

We, as of 2019, have reached that "Earth 0.99" first, and we know our "competitors", also with the 2019 degree of "sophistication", would be there too, in some 20 years — with evil goals of conquering or otherwise controlling that planet.

If our goal is to best help "them" (or "us" in 1969) to prepare to defend against such a threat from the "evil" 2019 humans, what would the most reliable strategy be?

Here I'm doubly lost. We could establish the Internet for them and teach them what Twitter is, for instance. And we could moderate it 100% to serve our own goals.

But what could those goals be, so that accomplishing them would more or less guarantee that they would be immune to an attack of the "an all-seeing evil fake-news-powered" one?

I can really only think of providing them with the technology to quickly set up some, well, Space Force, so what that very civilization would be able to physically defend itself from the Star Wars 1.0 type of attack.

Or perhaps just tell them exactly what's going to happen, so that instead of militarizing themselves those guys would prepare to negotiate with the prospective attachers. Granted, they would have to negotiate from weak grounds, but still they should be able to secure some wins for their society in exchange for whatever those "evil" guys could want.

The above alone, in my thought experiment, is a solid proof of how fragile the "human material" is, both in 1969 and in 2019. Because even if we could offer them Wikipedia plus all the modern and now-top-secret technologies, there really is no way to make sure those guys don't self-destruct "thanks to" them.

How soon do you think the deep learning bubble will burst? [1/2]

While I do believe in bubbles — and also in the fact that both the “AI” and the modern venture-backed “economy” are perfect examples of them — here’s an argument for why deep learning is not a bubble.

Neural networks are just convex optimization. Right? Therefore, they are not capable of solving any problems that have anything to do with AGI, or overall “intelligence”. Right?

Well. Right. But.

One. There is literally zero evidence that human “intelligence” is anything other than a weird and luckily well-functioning combination of heuristics and meta-heuristics of which paths to continue exploring and which drop as nor promising enough. Here’s a tweet that I wholeheartedly agree with: https://twitter.com/dmimno/status/949302857651671040

Also, it’s important to note here that human “intelligence” and human consciousness are two different things. I wholeheartedly respect the latter, and would be among the first to admit it’s quite some field to research — which puts me under no obligation to praise human “intelligence” as something unique or otherwise special.

Two. There is another form of learning which is about as powerful as neural nets: gradient boosting, and, specifically, gradient boosted trees. In a way, they can also be seen as a form of convex optimization (although a few clever tricks can easily make them “less convex” than neural nets). The main take home message here is that neural nets are nowhere near unique, they are just more noticeable for most people because when it comes to applications most people do notice — such as real-time video processing — it’s neural nets that run the party. Thus, the “power” of neural nets is not something miraculous that has no match in science and industry, but rather just one of the easy-to-demonstrate manifestations of how far did the machine learning algorithms actually advance by now.

Imagine the world where the printing press is just emerging: it surely is a massive change, but the printing press alone does not change the humanity the way we know it (or the way it used to be), it just a) brings out the best in people who are ready and capable of adjusting, and b) tanks those who prefer conservatism for the sake of conservatism. The humankind has seen those “threats” before, and none of them were existential.

Back to the point.

Yes, the “evil AGI” threat is real, much like any technology that can be used for destructive purposes, be it antibiotics, nuclear energy, or the Internet.

No, deep learning is not The Bubble, and neither it is a part of some A Bubble. What is overhyped is how deep learning is PR-ed and populated to most lay-people.

∼ ∼ ∼

There are certain rather specific fields where deep learning rocks. They are the fields where:

⒈The amount of data is massive.
⒉Quick, ideally, real-time results are desired.
⒊ The output (the “answers”) of the algorithms are fundamentally of probabilistic nature.
⒋Humans themselves are notoriously imperfect, to the degree that it’s not hard to beat the “better than an expert” bar.

For example, when it comes to speech recognition, or telling cancerous tumors from non-dangerous ones, or adding a realistically-looking moustache to one’s face as they are in a video call — all these are good examples of where deep learning can indeed do miracles, and overall appear as something revolutionary, as a technology that beats anything and everything we have seen before.

How soon do you think the deep learning bubble will burst? [2/2]

On a grand scheme of things though, quite frankly, deep learning has yet to leave a noticeable dent.

To get back to the bubble subject: I don’t even think there’s any bubble with neural nets these days, as the market is still not full. If you have a teenage kid who’s into computer science, pushing them towards learning how to operate neural nets is a relatively safe bet, as the industry still needs experts in the field, and there are not enough qualified people. Yours truly can be a good example here: I have never trained a deep network in my life, but my academia/mathematical knowledge of how they work, times my hands-on experience with “non-deep” machine learning, keep proving themselves useful to hardcore deep learning teams over and over again, simply because the nature of problems those teams are facing is still more of hands down engineering and common sense than rocket science mathematics or computer science.

Analogies are a terrible way of explaining things, but here’s one more: servicing cars. On the one hand, with electric vehicles taking over, the job of being able to fix the generator or to diagnose the gearbox are going to go extinct soon. On the other hand, they were and are here, for well over fifty years as far as I can tell, so that generations of people were successfully betting their careers on servicing cars.

If anything, becoming hands-down proficient with neural nets, given that we are still in the early stages of their adoption (think of all those custom-made chips built specifically to train and/or apply certain nets!), will likely remain a lucrative career for some 10 .. 15 years to come.

I would be among the first to argue that if and when we are close to the AGI, this particular knowledge will likely become rudimentary. But it’s not going to happen in the next ten, and, very likely, twenty years.

∼ ∼ ∼

So, with all my lack of interest in deep learning (so far), and with me admitting wholeheartedly that the field is hyped beyond comprehension, I am unable to believe deep learning is in the bubble now.

If you like it, dive in, and have fun!

Even though computers are getting faster, why does it barely keep up with most softwares written today? What makes softwares bulkier and slower these days and how can we fix it?

Never attribute to malice that which is adequately explained by stupidity.

Not by the stupidity of software developers, or of the managers of those developers, or of the executives of those managers. Don’t underestimate the stupidity of the average user.

And then remember we live in the world of capitalism, so the user is king.

See where I’m going with this?

People will buy new pieces of hardware. People will install new flashy apps. People will pay for those in-app merch. People will click on ads in those apps.

And people don’t want software that is not slow and not bulky. Evidently. People want software that does something they want or need it do to, and they are comfortable waiting for several seconds a time. Every. Single. Fucking. Time.

Yes, some people are not like this. But we are the minority. We use terminals and vi to keep things fast. We like it.

But we are not the customer who is paying.

Do you how does an average customer look like? Look no further. Here it is. Quod erat demonstrandum.

What "unpopular work advice" actually helped you in your career? Why?

Good question. After giving it a long thought, it’s not really an advice, but something that keeps proving useful over time.

Don’t push it.

When the task is creative, or otherwise requires some state of mind, there’s a good chance that trying to get it done “against the mood” would only make things worse.

Example tasks that are a perfect way to get oneself busy when the creativity sparks are out of fuel:

∙ File an expense report (yes, those suck).
∙ Book flights or hotels.
∙ Re-read and polish some instructions / documentation.
∙ Fix that broken test.
∙ Just go through some documents / manuals / drafts what you said you will go through, even if it’s lo-pri.
∙ Clean up some e-mail or task trackers or old bugs.
∙ Take a walk outside (not in the heat though, sorry, although I love working from Thailand too).

Example tasks on which, at least I personally, would most likely not be productive enough; to the degree that putting them aside even for several days would still be a net win once I get to them with the right peace of mind:

∙ Outline some nontrivial ideas in a presentable way.
∙ Implement a creative piece of an algorithm.
∙ Have an important conversation, be it technical or managerial.
∙ Test some machine learning idea for some corner cases to isolate potential issues.
∙ Schedule meetings conversation on a trip which I have yet to decide the duration and the time frame of.

∼ ∼ ∼

Another, and perhaps cleaner, way to look at the above is through the lens of the expectations of others we are looking forward to fulfilling, and through the lens of our own ego we work to fulfill by means of getting some work things done. Upon closer examination, neither of the two above is a good motivator.

Others expecting us to deliver by certain date and time are, in most cases, just projecting their own perception of the pace they would like to see you perform at. If that pace is uncomfortable, it’s virtually always better to indicate it up front rather than to try to “earn” their “trust”. And this is one of the aspects where a conversation could make things slightly worse, while just saying “it’ll be done by next week [implying: sorry, not today]”, and actually getting it done by the next week is a path to a healthier business relationship.

And one’s own ego, as the pride and satisfaction we enjoy after getting things done, is, while generally a very positive emotion to experience, just not a good enough carrot to make oneself work towards. It’s the end, not the means, at least in my book these days.

∼ ∼ ∼

Obviously, if one senses the urge to take a break and browse Instagram or Dilbert for several hours, it’s a lot better to make themselves focus on the tasks from the fist bullet-pointed list above.

The best advice that helps me quite a bit, while can be considered unpopular, is to not even try to get to the tasks from the second bullet-pointed list above until I am comfortably enough in the mood to get to them.

Do virtually anything else, as long as it is even mildly productive, and/or as long as it contributes to getting myself back into the flow of creativity in some reasonable time window.

Yes, this time window can be several days, or even “after this trip/conference take place” or “after that conversation that is next week is held”. Just don’t push it when the moment is not right.

∼ ∼ ∼

The above, certainly, applies to creative parts of jobs only. If you’re a pilot, or a dentist, or a bartender, then, chances are, your priorities are different.

But, trust me: Much like you may sometimes dream of changing your career into something more creative, quite a few creative people around, quite often, dream of having a somewhat repetitive yet rewarding job that you do now, as opposed to effectively being under the control of our immediate mood and depth of inspiration.

Why aren't full day programming interviews structured around a single real world project, each round focusing on one aspect of the project (system design, interface design, algorithm design, programming, testing)?

Here’s my now-controversial, albeit perfectly reasonable, view on this.

Because, despite what everyone keeps saying, programming interviews are not broken.

Let’s approach programming interviews rationally, as a case of game-theory-based negotiations, and otherwise as a form of a multivariate optimization problem. It’s simple.

∼ ∼ ∼

The employer (corporation, startup, school, etc.) wants to hire an engineer (intern, student, partner, co-founder) in order to accomplsh certain goals.

Speaking meritocratically (cisgender white straight male privilege and overall a prime example of problematic patriarchy, I know, I know), the employer primarily cares about three aspects of such a hire:

⒈ Increasing value of their venture thanks to the hire (be it revenue, valuation, connections, etc.),

⒉ Minimizing the risks associated with the hire (be it the risk of non-delivery, the risk of the team possibly not enjoying working with this person, or the reputational risks, including the risk of being mocked by SJWs or otherwise accused of something non-meritocratic by random morons), and

⒊ Making this hire as effectively as possible (because opportunity cost is real, and, for most positions, a good enough candidate tomorrow is far better than a perfect one a quarter later).

∼ ∼ ∼

Now, I’m about to throw in several assumptions. They reflect my experience of being an engineer and working with engineers for twenty years; they may, or course, change later in my life, but I’d bet they won’t get altered by 180 degrees.

The assumptions are:

∙ Talented people tend to be talented in many ways.

In other words, intelligence, both liquid and crystal, and both pure analytical and social, exists and can be measured, or, at least approximated.

And one full day of various conversations with different people gives the employer some 95+% of the information they need to make the hiring decision; 99+% as their references are checked.

∙ The cost of a bad hire is often not as high as the cost of no hire.

True, with modern SJW trends it’s dangerous to hire a minority (or just a woman) knowing well that the chance they will not fit well and may have to be let go some half a year down the road is over some point one percent — the second-order, non-meritocratic, risk of having to deal with BS consequences of such an unpopular decision are far greater than they have ever been.

Still, one’s skills set and work ethic required to get certain job done are the substance, that, in a well-functioning organization, is always more essential than the form of keeping the “optics” “clean”. So, healthy organizations simply are open to taking those risks.

∙ The cost of spending too much time on a possibly underqualified candidate is just not worth it.

The trend of doing a “micro-internship” for a day or two, to work alongside the candidate and see how well can they do, technically and as part of the team, is indeed emerging these days. I have done two of those (as a candidate, and a few myself as the hiring person), and I’ve enjoyed them greatly.

Still, it’s clearly a value-subtractor for the company, in my view, to proceed to this stage with a candidate the chances of wanting to hire whom are not some 98% at least.

And, with the desire to get the candidate this high, spending a day working with them, in a limbo state betwen evaluating them deeper and selling them on the idea to join our company, just means leaving value on the table — the value that could be realized, in different ways, should the candidate already be given a clear “yes” or a clear “no”.

∼ ∼ ∼

That’s about it. Very simple, if you ask me.

The human brain operates with models. Yes, all of them are wrong, but some prove to be extremely useful.

Different authors prefer different terms for those models: hypothesis, imagination, or delusion are used ofen. Regardless of how we call them, the very process of human communication — and human thinking, as they co-evolved — is the process of being able to:

⒈ Generate new models,
⒉ Validate and discard the "more wrong" ones, and
⒊ Explain existing models to other humans.

The last point has two distinct sub-points. The goals of communication are:

⒊⒜ To distill the model in the very speaker's head, and
⒊⒝ To make more people aware of this model (so that good models "evolve" within more brains, I'll probably write more about this later).

∼ ∼ ∼

I believe it only takes half a step from the above to solve the AGI and the explainable AI problems: The machine should involve the ease with which it can "train humans" in its fitting cost function.

That's it. Yes, so straightforward.

Here is an example implementation that looks plausible to me:

∙ Take a simple domain of problems. Say, geometry as Euclid saw it, or basic computer science as Knuth or CLRS teach it.

∙ Use the form of adversarial learning for the machine to simultaneously grow its knowledge on the domain, by postulating problems it can formulate but not yet solve, and by solving problems that it could not have previously solved. This approach was tried 50+ years ago, and it does yield tremendous results (example, from late 70s / early 80s). Using modern ideas such as GANs and RNNs, the fitting process can be extremely formal yet powerful.

∙ Now, add "regularization" to that model training: make sure the humans are involved. Take a class of volunteers and have the machine "explain" them a few problems that are currently "at the frontier" of this machine's knowledge. These volunteers, by the way, can be part of some large-scale MMORPG, but that's a separate, much longer, conversation.

∙ Thus, not only the domain of problems this machine would be able to postulate and solve will grow, but, together with growing the set of problems and the means to solve them, the machine will grow the knowledge of how to navigate the humans along the path of comprehending those problems and the methods to solve them.

∼ ∼ ∼

My intuition says that in the field of number theory, for example, starting from plain Peano arithmetic this { machine "learning" the world + humans "validating" its "explanations" } system could easily reach the concepts of multiplication and division, prime numbers, factorization, rational numbers, Diophantine equations, and so on. In the field of computer science I would bet sorting algorithms, the Dijkstra algorithm, spanning trees and maximum flows are the examples of what will be discovered automagically relatively quickly.

As a minor result, the above approach would likely result in paradigm shifts when it comes to how to teach the above concepts to humans. After having some model of the human mind within itself, the above machine would most likely be able to act like Socrates or Archimedes, postulating solvable problems of increasing complexity to the scholar, so that the they both enjoy their progress and are rapidly improving their skills set.

As a major result, well, I believe what the above would result in is indeed the AI that finds "satisfaction" in communicating with humans in the human language, helping us, humans, solve the problems we are presently facing.

The above may sound overly optimistic, but I seriously am open to believing we are this close to the form of "machine intelligence" that can help us in our daily lives.

A genie asks you to describe a programming language, which he will then create. What do you tell him?

Here’s what a software architect in me says:

⒈ Clean, short, and unambiguous syntax.

⒉ Easy to read and understand by human engineers.

⒊ Strongly typed, both at programming and at metaprogramming.

⒋ Can be both compiled (C++-grade performance) and evaluated on the fly (zero compilation time).

⒌ Not requiring an IDE to use effectively (“vi-friendly”). But, at the same time, it should be easy to use lightweight browser-based development tools, including interactive notebooks with visualizations, DB connectors, and other nice-to-have features integrated seamlessly.

⒍ Libraries and modules support without security holes or dependency hell.

⒎ An integrated build and package management system.

⒏ Cross-platform, of course.

⒐ Mathematics-friendly: to realize my dream of developers, production engineers, and scientists/mathematicians speaking the same programming language.

⒑ Network effect friendly: the language must enter this world in a way that makes more and more people use it.

And here is what a rationalizing philosopher in me says:

• It would be nice to have the language that can leverage modern technologies, such as AR/VR, when it comes to having people learn it, program in it, and collaborate using it. No clear idea of how could it look like, but IDEs with methods browsing, or code review tools, or linters could clearly become much better in 3D.

• It would be nice for the language to indeed be AI-friendly, so that tools such as Resharper would not only be useful, but potentially boundless in power, augmenting human engineers dramatically, leaving the creative part to the human brain, while hiding the complexity of the underlying implementation from this very human for as long as it is possible.

How would we know if our "AI" models have consciousness?

Before dismissing the question as too broad, think of an "easier" one: How could an alien super-intelligent civilization observing every atom of Earth today come to the conclusion that some processes in humans' brains are not merely functional, but also result in us experiencing qualia?

Even considering only today's approach to "AI", the deep learning one, I'm close to acknowledging that the problem of being unable to comprehend what happens in the inner layers of a deep recurring neural net is conceptually indistinguishable from the problem of an emotion-less super-intelligence possibly comprehending humans' inner selves.

Hackathons. I view them as a waste of time and energy.

It is not about length constraints. It is about the attitude. I don't enjoy building small things. I prefer the big ones to come from underneath my fingertips.

Big things are demanding.

• They require thorough thinking. In the order of hours, at least.

• They may require research. In the order of days, easily.

• They may require good understanding of what current and soon-to-come technologies are best to use.

• They require understanding of how and where do they fit the world.

• They often call for several conversations with many people. Who, in their turn, are often busy enough with their own big things.

Hackathons, the way I understand them, are the exact opposite:

• Doesn't matter if it does the real thing. It looks sexy. Ship it.

• Doesn't matter if it is well engineered. It somehow works. Ship it.

• Doesn't matter if it is about to be thrown away next week. It's not supposed to live long anyway. Now ship it.

• Doesn't matter if it does not sustain any constructive criticism. People who are capable of providing it would rarely show up for a hackathon. Those who do show up would usually have the attitude of ... yeah, right. "Ship it".

∼ ∼ ∼

I perceive the culture of hackathons to be a desperate attempt to find someone who can code at least something. Because those who can build real things are already busy doing that. Most of the people who remain uncertain are those unwilling to or incapable of thinking big.

True, a hackathon is a good way to gather those people together and have them do something useful. This community, however, is unlikely to interest me, or other people who want and can build real things.

∼ ∼ ∼

Furthermore, I find no emotional attachment to short-term results one can demonstrate during a hackathon.

• I don't need to see or touch "something that is live". My imagination is good enough to understand what is about to happen.

• Unless it's a major outage, whether it takes or day or a week is almost irrelevant. Whether it lasts for three weeks, three months or three years is what counts at the end.

• I care for long-term solutions and think of any "hacked" code as technical debt. To provide a contrast: I would rather spend a day writing a nontrivial test for a feature that is likely functioning right as it is right now. To be sure, for today and for tomorrow, because bugs do happen.

• I think on the scale of products. Not technologies and tools.

A day spent in talking with the core team and drawing prospective designs on the whiteboard is more productive than a day where "those two new technologies we always wanted to check out" got checked out.

Also, while I agree that a hackathon boosts overall efficiency on a scale of several dozen hours, I truly believe this "efficiency" is not created out of nowhere but is rather being "borrowed" from past and future productivity.

In other words, I belive that for a well-functioning team, the perceived "boost" in productivity over a weekend is generally a loss when considered holistically as part of a longer time interval.

And dedicating our time to something that is long-term a productivity killer, and, furthermore, staying delusional about the very sign of the contribution of this activity, is absolutely not what I want my team to consciously choose.

Hackathons [2/2].

As a matter of fact, what I have been working on for the past week (2014) is perfectly a non-hackathon. My goal was to have a real-time dashboard. A simple one, that could be put together in several hours. Instead, I dedicated three full days to testing the code that ensures the timestamps of the entries from potentially different soures are synchronized. And that the modules involved would restart in the right order should connectivity or power outage or other issue takes place.

And then to making sure the { HTTP <=> node.js <=> ssh <=> JSON to Thrift <=> C++ backend <=> Thrift to JSON <=> JSON to HTML } chain does what it should. By exposing quite a few hooks and endpoints along the way. And watching them in a few nontrivial situations I have artificially created.

Of ~60 hours of work, the numbers that ultimately make sense appeared in the past four. They started to look presentable enough to be projected onto a big screen (i.e., non-JSON) in the past one. This "slowness" did not bother me at all. I was not "itchy" to see how those things may look before making sure that the foundation on which the final solution is built is stable enough. I just wanted to build it right, and I did so.

∼ ∼ ∼

And, had the above not been done within those few days, I would have simply shot an e-mail to one or two people who depend on me, along the famous Blizzard lines of "it's ready when it's ready".

I wish more engineers would follow this paradigm.

Thoughts on Google potentially dealing an "API blow" to the ad blocker industry.

Technically, browsers don't change that quickly. Web standards change even slower. And most websites really do not care to even try to catch up.

Therefore, the "baseline strategy" for one to just keep using the browser they have today is not unwise. Chromium, for example, is open source. I believe, in its present shape and form, it is not constrained by restrictive Google licenses. And I bet enough enthusiasts will work to keep it available — for the general public, not only for the nerdiest of us.

On the short time scale, say, one to three years, the devil is in details:

• Security. It's not hard to keep the open version up to date with the industry. It would be a lot harder to keep the "obsolete by design" browser secure though, especially if Google decides to not release the newest patches openly. V8 is a pretty damn complicated beast, to say the least. Also, this opens doors to the next point, which is:

• Planned obsolescence. Large players may consciously refuse to fully support the "older" browser. They are, after all, capitalist entities just like Google, and they want to maximize their own profits, which largely come from ads. And "we care for the security of our users" would sound like a plausible justification to urge the users to "upgrade", should some New York Times decide to push towards it.

On a longer time scale, ten years and beyond, the situation gets a lot more interesting:

⒈It's unclear whether the browser will remain one's main gateway into the Internet. For many people Internet is Facebook. For many it's perfectly contained within the apps they use. The competition may even evolve so that companies like Medium would rather adjust their website to load well (and safely!) in Facebook's built-in "preview browser", than in some abstract "Web browser" application.

⒉It's unclear whether Google would retain the position of monopoly as the younger generation grows up. With all its imperfections, for example, I'm still a loyal GMail user. It won't change next year for me, but it may well change within the next ten. Forget me; if the younger people are choosing other e-mail providers, as well as other maps and search services, of which there are plenty, Google may be hard-pressed to actually take the user's side with the browser — keeping it open and ad-free-friendly — because the alternative would be to lose users even faster.

⒊It's unclear whether the very ad model would stay the way it is today. Subscriptions and micro-payments, so far, did not really threaten the model substantially. But I would not be so certain the world would be the same ten years from now.

⒋It's unclear whether the Web ten years from now would look same way as it does today. Some form of decentralization coming back is one remote possibility. Think of everyone online being their own hosting provider, and a lot changes, especially for the services whose business model is to connect billions of people and yield profits of them using their services. More likely optoins come in different shapes and forms, and include different devices, such as AR/VR, or goggles, or brain-computer interfaces. I, for one, am not sure the CPM/CPC/CPA format of ads would be very successful there.

On the grand scheme of things, this situation also happens to be a prime example of where a relatively simple regulation could deal a deadly blow to a corporation, purely to benefit the end user, with seemingly no side effects.

Hypothetically, if the European Parliament decides to fine Google around a billion dollars for every month during which Google did not release a 100% ad-blocking-compliant browser for free (assuming some enterprise, paid, version is available), I, for one, would have an elevated faith in the future of how IT regulation could help our world.

Relevant link: https://www.ubergizmo.com/2019/01/google-chrome-deathblow-ad-blocker-extensions/

On the value of idea vs. execution.

There is an important observation I am coming to a bit too often these days; often enough to declare it a life lesson of the past ten years: One perfectly done great thing just does not win over a hundred good ones.

Take businesses. Many successful companies trace down to "... and this was the right idea, at the right time, well executed". Such a generalization often appears true, but it is, at best, misleading.

A perfectly executed big idea at the right time is nowhere near a recipe for success. It is a good first step. It opens the first few doors. But that's pretty much it. The rest is hard work, a series of a lot more modest successes, and some luck along the way.

There is a decent chance that back in 1998 someone alongside Google has come up with the idea of PageRank. And, in the seventies and in the eighties, neither BASIC nor soon-to-be MS-DOS were outstanding enough to bet on right away. In fact, I think the very opposite is true.

We, humans, consistently make a mistake of glorifying the stories that intuitively paint the picture of "this is the spark that ignited it all".

Perhaps we are simply delusional about this inflated role that the "spark" plays. I would not be surprised if the nature of reality is exactly the opposite: Someone who is capable and ready to build and operate a business would do well, in their first venture, if they would begin with a modest, simple, idea; while those who can't even start, are extremely unlikely to succeed even if The Very Best Idea does come to their mind.

Perhaps instead of glorifying ideas we should learn how to pragmatically examine their utility? Instead of "just how big could it get", ask yourself a few simple down to Earth questions. Does this idea drive you? Good, you'll be working hard to make it happen. Do you enjoy talking about it? More power to you when it comes to hiring and/or to raising money. Is it in the space where you are an expert, have a name, and network? Well, that would sure in handy too. Approach it step by step and thou shalt receive. The rest is execution.

∼ ∼ ∼

Don't get me wrong, the secret sauce is essential. A hundred well-executed mediocre pieces would not sum up to become great. A hundred mediocre pieces won't attract the customers or investors, won't pass due diligence, and will most likely not be of high value in the first place.

It's just important to not focus excessively on the very secret sauce. It may well occupy 90+% of the work done in the first several months, on the prototyping stage. But if by year two, with no real customer, the main focus of the founders is how to perfect this secret sauce, something is very wrong.

Instead, it is most likely that leaving a few key people to maintain and improve the secret sauce, while focusing the efforts on what the business should really be focusing on — pleasing the customer — is a way better strategy, assuming the goal of the business is to ultimately succeed in the capitalistic meaning of this word.

∼ ∼ ∼

Moreover, interestingly, the above approach scales well to human relationships, both professional and personal. Those who are telling a story of "this person did X and I immediately knew we'd do great things together" are, likely, deluding themselves. Quite frankly, I can't think of anything extraordinary enough that if one successfully performs certain act, it is guaranteed to spark a long-lasting engagement with close to zero chance of falling apart.

Someone single-handedly saved the project by completing what seemed impossible just over the weekend? Extraordinary. Someone ran into a burning buliding to save a child? Fantastic. Sorry though, neither is a solid predictor for anything rather than "they would likely be able to do it again if necessary".

Look for that "spark", believing it ensures prosperity, and more often than not you would find yourself disappointed. Look for predictability and consistency over an extended period of time, and, voila: here is a professional to work throughout your career, or a loyal friend for life.

On humans vs. non-humans from the standpoint of consciousness [ 1/2 ]

The idea of a philosophical zombie, or p-zombie, comes up frequently in books and podcasts about consciousness. Simply put, a p-zombie is an entity that lacks consciousness, but acts, at least towards the humans, in a way indistinguishable from a conscious being.

Many thought experiments can be imagined by asking various questions about how comfortable would a human feel, or be compelled to act, in different scenarios that involve p-zombies.

Putting the personal emotional connection question aside, the subject matter looks straightforward to me when approached from the standpoint of how is our society presently organized.

Consider your own, or your relative's or your closed friend's professional life. Say, they are interviewing candidates, and the nature of their work is such as some people work remote. After a while the desired candidate is found, an offer is extended, and this candidate is ready to join next Monday.

Now, on a Friday evening you realize this soon-to-be-colleague is not a human, but a p-zombie.

They are a remote worker. You, and your team, will interact with them via e-mail, chat, and an occasional voice or video call. This candidate has passed the interviews, and is better than other candidates at getting their job done. Moreover, they are willing to work for less money, for a number of reasons, from willing to gain the relevant experience, to not requiring health insurances of any sort.

Would you feel uneasy about the discovery that this candidate is not a human?

To make the case even stronger, let's turn the tables. You, or your friend/relative, are looking for the next gig. There is an interesting opportunity, good team, and it pays well. As you are ready to sign, you are being told that the founder, or perhaps even your immediate manager, is not a human but rather a p-zombie. Would you reconsider?

∼ ∼ ∼

In the above scenarios my view is extremely simple. In both cases, paying any attention to whether your prospective colleague or a boss is not a human is plain that: racism.

By the very definition of the p-zombie, you, as a human, can not tell if they are conscious or no. In other words, your decision on whether to work what that entity or no, and, overall, whether to treat them as an individual or no, depends entirely on some artificial construct that some "p-zombie-detecting" device shows a green light or an orange one when queried about whether certain entity is a human or a p-zombie.

Nowadays, we detest the notion of refusing to work or otherwise deal on equal terms with people of different races, genders, eternities, sexual orientations, and many, many other dimensions.

There is even a new term "neurodiversity", to highlight that acting in a weird ways in otherwise common situation should be tolerated and/or ignored, as long as the actor is not doing it on purpose, but rather just happens to have their brain "wired" in certain unorthodox way. As long as their job is not directly affected by their unusual behavior, we are supposed to be inclusive and welcome them.

It has gone to the point where I was once explicitly told that being an active member of the Flat Earth society does not disqualify one from being employed as a data scientist, as long as their day job has nothing to do with whether the Earth is flat or no. Yes, these days we are supposed to tolerate the lack of critical thinking even to such degree, for the sake of being a diverse and inclusive society. Even for a job where we have all the reasons to believe critical thinking is absolutely essential, such as the job of a data scientist.