⚡️ Async/.await in Rust reached beta channel

The Rust crates ecosystem and async stdlib (https://async.rs) gradually moving to the more convenient syntax and introduce asynchronous paradigm where it fits at it's best, where previously are applied the composition of combinator-based futures.

https://blog.rust-lang.org/2019/09/30/Async-await-hits-beta.html

▶ And (even more) inspiring talk and presentation about bringing Rust into Linux kernel and modules development by Josh Triplett, the Intel principal engineer, at Open Source Technology Summit 2019, in late August this year:

https://youtu.be/l9hM0h6IQDo

https://hub.packtpub.com/rust-is-the-future-of-systems-programming-c-is-the-new-assembly-intel-principal-engineer-josh-triplett/

https://pcper.com/2019/09/mozilla-rust-considered-for-linux-drivers/

Discussion on LWN:

https://lwn.net/Articles/797714/

https://lwn.net/Articles/797828/

#Rust

ParaSail - Parallel Specification and Implementation Language

Microsoft Research lecture about ParaSail programming language by Tucker Taft, led language designer of Ada 95 standard, one of the GNAT compiler authors, member, language and compilers researcher in the AdaCore.

ParaSail is a parallel programming language (which uses work-stealing tasks to threads scheduler) with region-based memory management (static destructors, based on compile-time inferencing of scopes & lifetimes as region types), with Ada-like syntax (compiler toolchian also includes parsers for Java and Python like syntaxes for ParaSail), and as Ada and SPARK, it's predecessors, targeting on mission-critical systems and applications, for example air traffic control systems, rail-road and metro subway control systems, avionics, automotive systems, space exploration industry, defence systems and so on.

https://www.youtube.com/watch?v=bNWTRdAlwUs

https://parasail-lang.org

https://adacore.github.io/ParaSail/images/parasail_ref_manual.pdf

https://blog.adacore.com/author/taft

https://github.com/AdaCore


#PL
#Lang

Rust lifetimes, NLL, Polonius and cutting edge of region based memory management (RBMM)

It's curious that notable implementations of RBMM memory management paradigm includes Cyclone, ParaSail and Rust, especially with current NLL borrow & ownership checker, based on control flow and data flow lifetimes analysis in more low-level MIR code, and which ensures that a reference/pointer type cannot outlive the type of the object it points to, without necessary of explicit lifetimes declarations in vast majority situations.

Also more recent forefront, cutting edge research and development in more precise analysis of scopes & lifetimes inferencing as region types includes Polonius, the next milestone of Rust borrow&ownership checker — https://github.com/rust-lang/polonius

You can read many curious details about researching, development and implementation process of Polonius and RBMM subsystem in Rust from the Niko Matsakis series of blog posts:

http://smallcultfollowing.com/babysteps/blog/2018/04/27/an-alias-based-formulation-of-the-borrow-checker/

http://smallcultfollowing.com/babysteps/blog/2018/11/01/after-nll-interprocedural-conflicts/

http://smallcultfollowing.com/babysteps/blog/2018/11/10/after-nll-moving-from-borrowed-data-and-the-sentinel-pattern/

http://smallcultfollowing.com/babysteps/blog/2019/01/17/polonius-and-region-errors/

http://smallcultfollowing.com/babysteps/blog/2019/01/21/hereditary-harrop-region-constraints/

And series blog posts about design of current MIR based analisys for borrowed & ownership reference types checker — NLL (non-lexical lifetimes):

http://smallcultfollowing.com/babysteps/blog/2016/04/27/non-lexical-lifetimes-introduction/

http://smallcultfollowing.com/babysteps/blog/2016/05/04/non-lexical-lifetimes-based-on-liveness/

http://smallcultfollowing.com/babysteps/blog/2016/05/09/non-lexical-lifetimes-adding-the-outlives-relation/

http://smallcultfollowing.com/babysteps/blog/2017/02/21/non-lexical-lifetimes-using-liveness-and-location/

http://smallcultfollowing.com/babysteps/blog/2017/07/11/non-lexical-lifetimes-draft-rfc-and-prototype-available/

http://smallcultfollowing.com/babysteps/blog/2018/06/15/mir-based-borrow-check-nll-status-update/

http://smallcultfollowing.com/babysteps/blog/2018/10/31/mir-based-borrowck-is-almost-here/


Previous corresponded publications:
@technologique/1417


#Rust

AWS Open Source supports development of Rust project & community

Amazon Web Services Open Source division becomes one of the major sponsors, by providing sponsorship and delivering cloud resources, for the Rust programming language project and it's ecosystem (crates.io, docs.rs, etc.).

https://aws.amazon.com/jp/blogs/opensource/aws-sponsorship-of-the-rust-project/

#Rust
#Amazon
#AWS

Firecracker microVMs run-time

Previously, almost over a year ago, Amazon developed in Rust and open sourced a lightweight microVM run-time & monitor, based on KVM hypervisor, for AWS cloud infrastructure and FaaS/serverless environments of AWS Lambda/Fargate/ECS - the Firecracker project - and continue to use Rust in development of tools, performance/latency critical and real-time components (with guaranteed response time, like diagnostics logs and metrics aggregation tools) of the AWS infrastructure.

https://aws.amazon.com/blogs/opensource/firecracker-open-source-secure-fast-microvm-serverless/

https://aws.amazon.com/blogs/aws/firecracker-lightweight-virtualization-for-serverless-computing/

https://firecracker-microvm.github.io

https://github.com/firecracker-microvm/firecracker

https://github.com/firecracker-microvm/firecracker-demo

https://github.com/firecracker-microvm/firecracker-containerd

#Rust
#Firecracker
#VM #microVM
#containers
#clustering
#cloud
#Amazon
#AWS

WeaveWorks FireKube & Ignite

Some of companies and projects, like WeaveWorks, already caught initiative, as they sees and feels much perspective in micro VMs for the future of clustering cloud infrastructures, started to explore and uses abilities of contanerd to develop the integration tools between Kubernetes and Firecracker, like Ignite and FireKube:

https://www.weave.works/blog/firekube-fast-and-secure-kubernetes-clusters-using-weave-ignite

https://github.com/weaveworks/wks-quickstart-firekube

https://github.com/weaveworks/ignite

http://tech.paulcz.net/blog/future-of-kubernetes-is-virtual-machines/

#Rust
#Firecracker
#VM #microVM
#containers
#clustering
#Kubernetes
#cloud

Truly reliable, much useful, really comfortable (by usability) and powerful (by customizable feautures of CI/CD pipelining) Git and Hg hosting, created by SourceHut with UI minimalism in mind and powerful feautures in their heart (open sourced here - https://git.sr.ht/~sircmpwn/?search=sr.ht), currently in alpha stages in development — for someone who had tired of heavyweight UIs and average ordinary project hostings, and searching, looking for something good and new.

https://sourcehut.org

https://git.sr.ht - for Git repos

https://hg.sr.ht - for Mercurial repos

#SCM
#Git
#Hg

🧨 GOTO Amsterdam 2019 :: Firecracker for serverless @ AWS

Great talk and presentation about Firecracker on recent GOTO Amsterdam 2019 conference from guys, who are closest to the project at Amazon — Radu Weiss (Software Development Manager at AWS, Firecracker team member) and Arun Gupta (Principal Open Source Technologist at Amazon Web Services), who's joined in the second part of the speech.

https://youtu.be/oKKOBsqQndY

#Rust
#Firecracker
#VM #microVM
#containers
#clustering
#cloud
#Amazon
#AWS

Links to the previous posts:
@technologique/1426

Breakthrough and crucial leap forward for Rust and Tokio

This November was really exciting for the Rust community!

async/.awat (new syntax for asyncronous code execution with zero-cost futures under the hood, which is in turn, correspondingly, based on epoll model of passive calls to handler functions by arisen events, i.e. based on more cost effective, truly zero-cost, lazy evaluation model of computations and state changes) and NLL (non-lexical lifetimes, new borrow & ownership checker, based on control flow and data flow lifetimes analysis in more low-level intermediate MIR byte code, not the syntax tree analysis as it was before) reached the stable state/branch and released in stable 1.39 compiler version!

https://blog.rust-lang.org/2019/11/07/Rust-1.39.0.html

https://blog.rust-lang.org/2019/11/07/Async-await-stable.html

https://blog.rust-lang.org/2019/11/01/nll-hard-errors.html

So, thus, in the next couple of months we will see how will increased the user base of Rust, the networking projects diversity in ecosystem and how the community will grow up in the near future due to this essential features, finally delivered into stable branch in these last two months.

Here is the really good articles, which describes how the futures, the "handler pyramid of doom" and it's asynchronous destructors are works and optimized under the hood of async/.await:

https://tmandry.gitlab.io/blog/posts/optimizing-await-1/

https://tmandry.gitlab.io/blog/posts/optimizing-await-2/

https://boats.gitlab.io/blog/post/poll-drop/


The Tokio had reached the 0.2 version!

https://tokio.rs/blog/2019-11-tokio-0-2/

Why it's so important? Just take a look at the new work-stealing thread scheduler (M:N thread mapper, which uses work-stealing model for more optimal CPU cores load and maps program asynchronous threads to system threads pool, dedicated to execute the program, which is in turn managed by the kernel and mapped to CPU cores, accordingly) in Tokio - that's a huge step forward! Initial testing using Hyper (raw http library) shows a 30%+ speed up!

https://tokio.rs/blog/2019-10-scheduler/

https://github.com/tokio-rs/tokio

We expect some movements up in the tournament table of the TechEmpower continuous frameworks benchmarks, so watch the results here:
https://tfb-status.techempower.com

Previous posts in a thread:

@technologique/1423

@technologique/1417

@technologique/1418

@technologique/1406

#Rust

For those who is still doesn't know how to monetize their open source project, or really want to be an open source developer (working remotely, with moving/location freedom, have independence from employers, have a free time for family, good work/life balance, and other good benefits for life), but also want to be a payed developer — I think this platform suits well for you and many others, for the great good!

https://xscode.com

#opensource

Actix-web was wiped out from GitHub by its original author

Something really sad and unhealthy in open source community happened yesterday with one of the most exciting projects in Rust - with Actix-web framework actually!

https://words.steveklabnik.com/a-sad-day-for-rust

This happened mainly because dialogue and decision making process went in a wrong way, and the escalation took a wrong turn in this situation.
So, thus, author decided to take his will and made a most hard (and not good for the Rust community) decision to wipe project repo out from GitHub.

This is the point when all we must to take a closer look at the situation, figuring out what actually happened (without any judgement) and realize that something wrong, unhealthy and really terrible happening, especially when community growing fast and in the wrong direction, when we could find that we are already on a point in a wrong way, maybe on a point of no return.

IMO, author did this to gain and attract more closest attention to a situation with less qualified third-party community from Reddit and it's participation (nowadays mostly as angry critiques, trolls and haters of Rust) in life & growth of Rust community projects, and to gain and attract attention to old and really controversial situation with (more or less justified) often usage of unsafe blocks in projects development as escape hatch from the guarantees of substructural type system and shared XOR mutable invariant of borrow and ownership checker in the Rust compiler.

Most of the participants in conversations doesn't really read Rust Internals forum (https://internals.rust-lang.org), and really couldn't figuring out that unsafe is an escape hatch (to using raw pointers, for example) and this is really essential for Rust as a systems programming language, 'cause not every situation, task and program could have a good fit into model of shared XOR mutable invariant (especially imagine creating of double-linked list with mutable elements, for example).
Nevertheless, unsafe doesn't disable in a whole the main invariants of substructural type system of Rust, it's just weakening the hard restrictions of type system where there're no other possibilities in task resolutions (although, programmer always must be aware of misuse and malpractice of unsafe hatch), this gives the flexibility for systems development (for creation of OS kernels, compilers, for example) and in a more precise way gives the powerful possibility for localisation of errors and undefined behaviour in program code.
Answer yourself honestly, could the developers in any other (and more danger!) programming language (except Rust, Ada and ParaSail) find, localize and fix the undefined behaviour in such huge project (as Actix-web was), as fast as Rust developers did this for that unsafe code block in Actix-web, which has led to undefined behavior and have had such insulting reaction of the haters of Rust, huh?
I think even with static analyzers and all that sanitizer techniques even experienced C++ developers couldn't localize and fix undefined behaviour cases in the compiler or any other huge program in a such fast and precise way (but, definitely and unfortunately most of such malfunction programs could be compiled in other languages, but not in Rust!), but unsafe in Rust gives this opportunity to developers, to get this work done as fast as possible!

And Steve Klabnik previously was already also write about this in one of his blog posts (https://words.steveklabnik.com/you-can-t-turn-off-the-borrow-checker-in-rust), and Alexey Kladov (https://github.com/matklad) also told about this and describe the use and misuse cases of unsafe in his lectures upon Rust on YouTube!

#Rust

Links to previous posts on the same subject:
@technologique/1332
@technologique/1333
@technologique/1334

Definitely, Actix is one of the best, and the most fastest and optimized frameworks for network services development. It's work-stealing scheduler for thread-mapper/multiplexer in fact is the best of the best, the brilliant piece of engineering!

🎗PS: A kindly and friendly reminder:

Whatever are doing and happening – all for the great good!

There're always exist solution and exit in every, even complicated, situation, even like this one!

And no matter what, you never ever should be disappointed in open source (and particular projects) power and it's values, be sure!

The power of open source is that we always could get a fork of a project, build a healthy community of co-thinkers and continue development of a good and useful product on a new level!

And, definitely, not only authors, but also every community member must be responsible and have a constructive position!

#opensource
#community

Author (Nikolay Kim) found a constructive and positive resolution for situation with Actix!

https://github.com/actix/actix-web/issues/1289

Artificial Intelligence (AI) podcast by Lex Fridman, in conversation with Jim Keller

Conversation with one of the most successful, experienced and absolutely brilliant hardware engineer and CPUs microarchitectures & architectures designer of our times - Jim Keller, the guest in AI podcast with Lex Fridman.

https://youtu.be/Nb2tebYAaOA

His contributions to CPUs architectures and microarchitectures design, and first principle hardware engineering, are really hard to overestimate — AMD K7, K8, K12, Zen (and the following generations - Zen+, Zen2, Zen3), Apple A4 and A5 SoC, he also a coauthor of AMD x86-64 instruction set, AMD HyperTransport interconnection system bus, worked on Tesla Autopilot Hardware, and finally he joined Intel, when they are reach a deadlock in development new CPUs of next generations and were in a loop of constantly emerging vulnerabilities in Intel CPUs architectures.

https://en.wikipedia.org/wiki/Jim_Keller_(engineer)

My personal impressions - this is the one of the top episodes of AI podcast, and surely, it's the most wonderful, informative, curious and interesting conversation in AI podcast for a long time!
Definitely highly recommend to watch and to listen - you could uncover for yourself again how modern computers and CPUs work under the hood from the first-class engineer who works on CPUs architectures for over a lifetime!

Previous posts on the same subjects:

@technologique/28

@technologique/1102

@technologique/1289

@technologique/1433

Also Jim states that Moore's Law isn't dead yet and this is heavily bonded to the progress in development of GPUs and AI specific machine learning acceleration ASIC chips/microprocessors, and here's the couple of lectures by Jim about these things:

https://www.youtube.com/watch?v=Qnl7--MvNAM

Some notes to my personal experience in interviewing processes and how it should be like

https://telegra.ph/on-interviews-02-09

I must to say directly that I really doesn't like a toxic interviewing processes, that are really stressful, especially such technical interviews, with online coding and stupid obvious questions ("what's the difference between abstract class and interface?") or otherwise with really advanced questions ("show me in code how we could optimize and which optimizing strategies we could use for upward and downward funarg problem in cactus-stack, which consist of call-stacks, for storing the context of suspended stackful coroutines? could you please show me your abilities and write a "simple" state machine (state automata) right here and right now for stackful coroutines suspension?" - and this ones is a real example of questions for Rust, C++ and Go interviews in which I had experience of participation!) — 'cause I experienced a lot of such interviews, and had failed on hundreds of them, despite I really confident with my knowledge on each interview! And this always was really stressful, not only on an interview itself, but mostly after, because I started to doubt in myself, my knowledge, experience, skills, abilities, and so on. This was isn't right, because when we experiencing stress we couldn't thinking right and make a right decisions. This is also a very thin psychological and physiological phenomenon, especially for creative minds and specialties.

I was a team lead (raised/growed up to this position mostly) in three companies in my career - at a fin-tech project of mobile payment system in a banking industry (where I started as a team-lead immediately at the beginning), and in two companies who targeting their products for cloud managed networking infrastructure for last mile mesh networks and cell operators - and in a team lead position I was managing teams (also working as a product owner in scrum agile task processing), hiring and examining new developers by myself, and had a deep knowledge of what exactly the interviewing process should be like, how it should be arranged and built, to give a confidence and calm conveniences for the interviewee, candidates to developers in a team, to be confident in their knowledge, and show themselves in a right way, 'cause they are just couldn't make a good impression to an interviewer twice!

Here is the good article about this subject - https://danluu.com/algorithms-interviews/
And I completely agree with every word of Dan Luu!
Most of the questions, tasks and coding samples on most of my interviews doesn't have any relevance to the further real work!
Most of the interviews are completely wrong and for me personally most of them shows the wrong arrangement processes that built in a teamwork and shows a bad toxic corporate culture in a companies. And I happy that I didn't got a work in and with such companies!
And that's why not only the candidates failed at the interviews, but also most of the companies, startups and products are failed in further - 'cause they choose a wrong approaches and started to build processes in a team relying on these wrong approaches. And they doesn't even have a clue about it! (And that's why such specialties as scrum-master existing!)