Thoughts for 2019
It's always interesting to look at the year ahead in technology, and so, some thoughts about 2019! Like 2018, these are less predictions, more extrapolations of what's already happening and what the second order effects might be. This year, thoughts on the economy, machine learning, testing, programming environments, http, open source, and at the end, a reflection on 2018’s thoughts.
Contraction. 📉 The tech sector may see a slowdown in 2019. There are a number of factors. Mobile phone growth is close to saturated. China appears to be slowing down, and with debt. The EU and United States are certain to see political turbulence. Cryptocurrencies are over as a speculation/wealth transfer vehicle, implying back to basics on product/market fit for the technology. Finally, while users will continue to trade privacy for convenience, social/adtech companies don't appear to have a good grasp on what to do next, such that further regulation or antirust seems wholly viable - and if ad spend sneezes the Web is likely to catch a cold. Despite all this, I’m not overly concerned about the macro economy, but am interested in effects on technical matters, which I think will result in a focus on bottom line activities like efficiency, step reduction, engineering fundamentals, and less bets on the margin. A contracted market should also mean more judicious investment theses by VCs (i.e., fewer, better bets). And some combination of Lyft/Uber/Slack/AirBnb should IPO, which will lift things.
Machine Learning. 👾 Last year I said “Machine learning as a general software skill.” and was a tad optimistic (see the recap below 😀). This year, I have a grab bag of observations. First, I expect PyTorch adoption to grow in 2019. The project’s doing a good job understanding end users and PyTorch + Caffe2 + Onnx is an attractive combination for production systems and ML pipelines. Speaking of pipelines, we’ll see a focus on systematic engineering approaches (eg using technologies like containers). With increased adoption companies will experience challenges running multiple models that demands codification; another consequence will be greater awareness of the limitation of notebooks. Cloud offerings are still iterating, but I’m hopeful there will be more and better options especially for training workloads. Machine learning is going to end up in everything, and 2019 will see the product management discipline learning how best to wield its automation and augmentation capabilities. As for the domain itself, I think we’ll see some applications of ML beyond supervised techniques where so much progress has been made, towards reinforcement learning and control theory. First, gathering and managing datasets for supervised training is expensive, even as ML winds up everywhere. Second, researchers are becoming interested in complementary approaches (eg if LeCun’s Learning World Models is indicative of a trend); as a result related tooling support should continue to appear as open source. Finally, these techniques fit conceptually with how online businesses increasingly need to build their products, as an infinite series of experiments and bets.
Testing in Production. 😎 Practice and maturity models are often lagging indicators of technology. Doing the majority of testing ahead of production, and especially the use of standalone integration environments is a fine example of best practices formed around older technical approaches. It’s a topic I think will see increased interest and discussion in 2019 due to changes in how we develop and deploy software becoming more established. Microservices and CI/CD have been around for the best part of a decade; QA has been folding into production monitoring and healthchecking for some time. Testing will go the same route - counterintuitively, this will improve the quality of systems. Newer approaches like Serverless, meshes and PWA change the nature of how software gets shipped - feature flagging, an even older technique seeing renewed momentum. These all further amplify a move to testing in live systems.
Integrated Browser Environments. 🤨 This one’s maybe already happened, or happening, and I’m the last the know. The theory is, if you can do some software engineering in a browser now, you will eventually do all software engineering in a browser later. We won’t all be doing this by the end of the year: on the other hand we’ve been doing real office work in browsers for some years, which was once considered unfeasible - something I would have agreed with for programming until recently, but now I think I’m very wrong on this point. I’ve had poor experiences with Electron apps in the past, but over the last year Visual Studio Code has became my default text editor and has forced me to squarely rethink what can be done with a V8/Chromium runtime. There’s repl.it, which is just programming in a browser, where “just” means an impressive 50 or so programming environments, and RunKit, which lets you write Node in a browser; effectively providing a much better notebook environment. These are not new products, but the underlying browser and deploy/packaging capabilities seems close to good enough at this point. Add in Serverless and typed functional programming, which by their nature abstract away from the things that have you and I using a “real” computer for “real” programming. This all affords a roadmap to make programming in browsers a reality. Github actions is an example of one direction things could take, relpl.it’s multiplayer feature is another. It’s no coincidence I mention GitHub and VS Code - Microsoft is well positioned to benefit from broad changes in software engineering practices.
QUIC and HTTP/3. 🤔 HTTP/2, originally SPDY, tackled issues with HTTP/1.x notably through its use of binary framing, bidirectional/multiplexed streams, and better compression options. It was a solid improvement albeit an something of incremental one, despite major changes in the wire protocol. HTTP/3 is different, it runs on QUIC, which uses UDP rather than TCP/IP. UDP provides benefits to connection start times and roundtrips relative to TCP/IP. QUIC also has further optimisations for negotiating encryption. Like HTTP/2, HTTP/3 will provide streams. QUIC has its own packet loss recovery approach to emulate what’s lost by not using TCP/IP, but does so per stream which reduces head of line blocking issues (I suspect significantly, but have no data). Now, HTTP/3 has a ways to go, so I’m not sure how much change we’ll see this year but I think this is one to watch. It feels like a more fundamental design shift in HTTP, one for example that might afford better options to leverage the push techniques introduced in HTTP/2.
Open Source. 😐 2018 resulted in a number of reactionary licences ostensibly focused on cloud vendor dominance. I don’t see this as anywhere near settled, despite recent analysis drawing parallels to early open source history and battles with proprietary software from the mid-aughts (in short: these are vanity licences, we’ve seen this play out before, no-one’s entitled to a return on investment). Cloud isn’t anything like proprietary software. There’s arguably been no strong thesis advanced to date on how so much value in service revenue can be tied up by a handful of vendors while companies formed around the projects and their founders clearly believe they are being squeezed out, without impacting the projects - you build and maintain it, we pay them to run it. On the other hand, given the level the world now depends on open source, existing community and maintenance models don’t seem sustainable. Empirically, there are structural issues and 2019 will be a year of increased discussion around incentives .
When I read back over the above, this year feels different in the sense that these are more like the onset or the consequences of fundamental shifts than specific, point-wise technical matters, especially around software development itself. I haven’t felt this way about my discipline for some time, and I’m very curious to see where 2019 takes things.
And last, a recap on 2018's thoughts:
Machine learning as a general software skill. There are signs but on reflection too early and obviously this takes time to work itself through. What I did see is less scepticism around machine learning, and thankfully, far less talk about artificial intelligence.
Serverless becomes acceptable, kind of. I underestimated this. Serverless is seeing strong adoption. In 2019, I’d expect broader realisation Serverless is a shift to a new development paradigm.
An orchestration winner won't settle down the cloud. Activity moved to service meshes as expected. In 2019, expect much talk of control planes and data planes as offerings try to find differentiation in application management layers rather than cluster orchestration.
Kotlin goes mainstream as a better Java. The most interesting datapoint I've seen is from Pusher, indicating Kotlin adoption is strong in the 2-5 year experience demographic, suggesting it's tracking developer growth, at least as important as its position on Redmonk or Tiobe.
Designers start to establish new interface principles. I was quite wrong here, considering my focus on Voice and AR. Voice is still voice. AR's exciting but we don't seem to know what to with it yet. On the upside, there’s been progress on Bot design due to a more realistic take on natural language capabilities; they look much more effective when designed to direct users through choices.