NewsTech News

Software Engineering Daily

softwareengineeringdaily.com

Software Daily: The world through the lens of software.

Play

Open in

2,221 episodes
Avg 53 min
English

Episodes 2,221

Newest Oldest Longest

All20 Unplayed20 In progress0 Played0

Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.

#1173
Jun 28, 2019 · 1 hr 5 min
Citus Data: Founding to Acquisition with Umur Cubukcu
A new database company needs to solve numerous problems in order to succeed. There are already lots of existing database companies, so a new company needs to find a way to strongly differentiate itself. Databases are core infrastructure, so a new database company must earn trust with its customers. A database is a complicated distributed system, so a database company must have strong engineering to get its product to market. Citus Data was founded in 2011. Around that time, companies were looking for solutions to both their online transactional workloads and their large-scale offline analytics workloads. Citus Data built its company around Postgres, a popular database that has been around since 1996. Products from Citus Data include a Postgres scalability extension as well as a cloud-hosted offering for spinning up Postgres instances. Citus Data was acquired by Microsoft earlier this year. In today’s episode, Umur Cubukcu joins the show to talk about the story of the company from its early days to its eventual acquisition. Umur describes the landscape of data systems in 2011 when the company started, and explains how that evolved to the current ecosystem. Umur also talks about how to make an acquisition successful and gives some perspective on the future of data platforms.
#1172
Jun 27, 2019 · 58 min
Lambda School Education with Andrew Madsen
Programmers are in high demand, and software engineering is a career path that is fun, creative, and lucrative. There are many people who want to transition into a career in software and are looking for the right path toward writing code. The traditional college computer science curriculum teaches some software engineering skills, but the time and financial cost of attending a university is prohibitive for many people who want to learn to code. Over the past decade, there have been several new models for software education. Online video platforms such as Udacity and Coursera put computer science courses online to be watched at the viewer’s convenience. Online schools such as Free Code Camp allow someone to learn how to program without any experience, and with no financial payment. Boot camps with income-sharing agreements such as App Academy create an in-person education environment that mimics a university, but with better cost structure and incentive alignment. Lambda School is an education system that takes elements of other software education models and combines them with newer SaaS technologies such as Slack and Zoom videoconferencing. Lambda School is an online software engineering curriculum with an income sharing agreement. “Income sharing agreement” means that the student does not pay for their education until they get a job. With this model, the student can pay back Lambda School after their software engineering education gets them a high-paying software engineering job. Andrew Madsen works at Lambda School, and he joins the show to describe the path that a student takes through Lambda School, the school’s curriculum for software education, and how Lambda School differs from the other options for coding education.
#1171
Jun 26, 2019 · 48 min
Afresh: Grocery Store Software with Volodymyr Kuleshov
A grocery store contains fruit, vegetables, meat, bread, and other items that can expire. In order to keep these items in stock, the store must be aware of how much food has been sold and what has gone bad. When a food item is low in stock, the store needs to order more of that food from a central distribution system. Managing food inventory is not simple. Some kinds of meat might expire faster than others. Avocados do not become ripe at the same rate as apples. In order to keep the shelves stocked, there are manual workflows for checking the inventory and ordering new inventory. Afresh is a company that builds software for grocery stores. Afresh works with grocery chains that have a central distribution center. These grocery stores already have some software. At the back of the store, inventory management systems maintain records of the items that the store has on the shelves. At the front of the store, checkout systems detect what has been sold and help to update inventory. When the inventory is running low, the store can order more inventory from the central distribution center, so that trucks can deliver more inventory. Afresh improves the operational intelligence of these stores by detecting spoilage among items that are prone to expiration, such as fruit. Volodomyr Kuleshov is the CTO and co-founder of Afresh and he joins the show to discuss the technical challenges of a grocery store, and the software that Afresh is building to make groceries more intelligent.
#1170
Jun 25, 2019 · 53 min
Airflow in Practice with Chaim Turkel
Apache Airflow is a system for scheduling and monitoring workflows for data engineering. Airflow can be used to schedule ETL jobs, machine learning work, and script execution. Airflow also gives a developer a high level view into the graph of dependencies for their data pipelines. Chaim Turkel is a backend data architect at Tikal. He joins the show to discuss a case study of using Airflow to rearchitect the data engineering workflow of a complex financial application. We discussed the problems that Airflow solves and the process of porting existing workflows to Airflow.
#1169
Jun 24, 2019 · 57 min
Project Management with Kurt Schrader
Software projects are organized and planned using project management software. Examples of project management software include JIRA, Trello, and Asana. There are hundreds of tools for managing a software project because there are infinite ways that a project could be managed. Google Docs changed project management by allowing documents to be easier to share and collaborate on. Newer SaaS tools such as Slack, Dropbox, and Notion have taken the design lessons from social networking apps to make enterprise software more engaging. As the tools improve, our project management strategies change, and new software tools emerge to fit those new management strategies. Kurt Schrader is the CEO of Clubhouse, a project management tool for software engineers. Kurt joins the show to talk about the history and future of project management tools. He also discusses the engineering challenges of improving performance on a complicated webapp. Project management tools often have to load hundreds of small objects on a page, which required performance optimizations in the Clubhouse frontend JavaScript library.
#1168
Jun 21, 2019 · 54 min
Niantic Real World with Paul Franceus
Niantic is the company behind Pokemon Go, an augmented reality game where users walk around in the real world and catch Pokemon which appear on their screen. The idea for augmented reality has existed for a long time. But the technology to bring augmented reality to the mass market has appeared only recently. Improved mobile technology makes it possible for a smartphone to display rendered 3-D images over a video stream without running out of battery. Ingress was the first game to come out of Niantic, followed by Pokemon Go, but there are other games on the way. Niantic is also working on the Niantic Real World platform, a “planet-scale” AR platform that will allow independent developers to build multiplayer augmented reality experiences that are as dynamic and entertaining as Pokemon Go. Paul Franceus is an engineer at Niantic, and he joins the show to describe his experience building and launching Pokemon Go, as well as abstracting the technology from Pokemon Go and opening up the Niantic Real World platform to developers. Niantic developer contest finalists
#1167
Jun 20, 2019 · 52 min
WebAssembly Compilation with Till Schneidereit
WebAssembly allows for web-based execution of languages other than JavaScript. Programs written in Rust or C++ can be compiled down to WebAssembly and shipped over the browser for on-the-fly execution in a safe, memory controlled environment. WebAssembly has been in development for more than two years, and is still an immature ecosystem because building the necessary tooling for WebAssembly is hard. Much of the web has been built around JavaScript and the V8 JavaScript engine, which has been tuned to optimize an interpreted language (JavaScript). WebAssembly modules are often written in C++ or Rust, which are compiled languages. There are engineering challenges at the edge between the interpreted JavaScript runtime and the precompiled WebAssembly modules. Till Schneidereit is a senior research engineering manager at Mozilla. He joins the show to discuss the compilation path of WebAssembly and the state of the ecosystem.
#1166
Jun 19, 2019 · 56 min
Infrastructure Wars with Sheng Liang
Sheng Liang was the lead developer on the original Java Virtual Machine. Today he works as the CEO of Rancher Labs, a company building a platform on top of Kubernetes. Sheng joins the show to discuss his experiences in the technology industry. The container orchestration wars had many victims. The competing standards for how an enterprise should manage its numerous containers caused several companies to go down a path where they were building infrastructure which eventually had to be replaced. As Sheng discusses in today’s episode, the container orchestration wars almost killed his company. Rancher was originally built on top of a different container orchestrator, and the migration to Kubernetes required a massive rewrite of the Rancher platform. The container orchestration wars were not the first technology battle that Sheng has seen in his career–and it won’t be his last. In today’s show, we discuss the nature of technology wars. Are they necessary? How can a software company minimize the damage caused by a war between competing standards? Sheng was an excellent guest and we didn’t cover nearly as many subjects as I wanted to, so we will have to do another show in the future!
#1165
Jun 18, 2019 · 1 hr 5 min
Kubernetes Operators with Rob Szumski
Kubernetes has made distributed systems easier to deploy and manage. As Kubernetes has become reliable, engineers have started to look for higher level abstractions we can define on top of Kubernetes. An operator is a method of packaging, deploying, and managing a Kubernetes application. Operators are useful for spinning up distributed systems such as Kafka, Redis, or MongoDB. These data systems are complicated, stateful applications with lots of failure domains. The operator framework enables a developer to deploy one of these complicated applications with less fear of the system crashing, or entering an erroneous state. Rob Szumski is an engineer at Red Hat. He joins the show to discuss Kubernetes, the operator pattern, and his time at CoreOS, which was acquired by Red Hat.
#1164
Jun 17, 2019 · 1 hr 16 min
Render: High Level Cloud with Anurag Goel
Cloud computing was popularized in 2006 with the launch of Amazon Web Services. AWS allowed developers to use remote server infrastructure with a simple set of APIs. But even with AWS, it was still not simple to deploy and manage a web application. In 2007, Heroku launched a platform built on top of AWS. Heroku focused on the developer experience by optimizing for users who were deploying Ruby on Rails applications. Since then, Heroku has expanded into other forms of managed infrastructure, including other application frameworks like NodeJS, and databases like Postgres. Heroku was the first popular “layer 2” cloud provider. Twelve years later, it is probably still the most popular. But there have been many other cloud providers built on top of AWS, including Netlify, Zeit, Spotinst, and Firebase. Layer 1 cloud providers are Google Cloud, AWS, Azure, Digital Ocean, and other raw infrastructure providers. These companies provide a great service in their low cost, commodity infrastructure. But the layer 1 providers are not optimizing for developer experience. They need to cater to a broad set of developers, some of whom want to work at a low level. A layer 2 cloud provider can build an opinionated solution that serves a subset of the overall cloud market particularly well. Render is a layer 2 cloud provider that optimizes for specific developer workflows, such as deploying a NodeJS web server, a static site, or a Docker container. Anurag Goel is the founder of Render, and he joins the show to discuss the strategy and the economics of Render. Anurag was also one of the early employees at Stripe, and he discusses his experience and learnings from working at the company.
#1163
Jun 14, 2019 · 1 hr 7 min
Elegant Puzzle with Will Larson
Software engineering is an art and a science. To manage engineers is to manage artists and scientists. Software companies build practical tools like payment systems, messaging products, and search engines. Software tools are the underpinnings of our modern lives. You might expect this core infrastructure which modern humans rely on to have been constructed with pure formulaic rigor. But the best software tools are not built within a totally defined process. Software is built through messy iteration. When a piece of software looks pristine, that is often a function of how many mistakes have been made, and then subsequently corrected for. There is no fixed process for how to build good software. As our tools get better, we have to update our software engineering practices to utilize those new tools. We have to rethink the style that we are working in. We have to discard old tools and procedures in order to pick up the new ones, and have higher leverage. As an organization scales, the structure of the organization needs to be modified. Team members need to be reallocated. Checks and balances need to be put in place. Rules and cultural practices need to codified, because a larger organization cannot have ties broken by an individual. Software is built by humans, and every management decision must be considered in the light of human psychology. When we change a line of code, the code does not get emotional about being altered. But the same cannot be said of humans. Even a minor conversation between an engineering manager and a direct report can have lasting implications. Will Larson is the author of An Elegant Puzzle: Systems of Engineering Management. He works on Foundation Engineering at Stripe, and has worked in engineering management at Uber, Digg, and other software companies. Elegant Puzzle provides strategies, tactics, and ruminations on software development. Will joins the show to explore the multifaceted subject of engineering management.
#1162
Jun 13, 2019 · 1 hr 5 min
Stripe Machine Learning Infrastructure with Rob Story and Kelley Rivoire
Machine learning allows software to improve as that software consumes more data. Machine learning is a tool that every software engineer wants to be able to use. Because machine learning is so broadly applicable, software companies want to make the tools more accessible to the developers across the organization. There are many steps that an engineer must go through to use machine learning, and each additional step inhibits the chances that the engineer will actually get their model into production. An engineer who wants to build machine learning into their application needs access to data sets. They need to join those data sets, and load them into a machine (or multiple machines) where their model can be trained. Once the model is trained, the model needs to test on additional data to ensure quality. If the initial model quality is insufficient, the engineer might need to tweak the training parameters. Once a model is accurate enough, the engineer needs to deploy that model. After deployment, the model might need to be updated with new data later on. If the model is processing sensitive or financially relevant data, a provenance process might be necessary to allow for an audit trail of decisions that have been made by the model. Rob Story and Kelley Rivoire are engineers working on machine learning infrastructure at Stripe. After recognizing the difficulties that engineers faced in creating and deploying machine learning models, Stripe engineers built out Railyard, an API for machine learning workloads within the company. Rob and Kelley join the show to discuss data engineering and machine learning at Stripe, and their work on Railyard.
#1161
Jun 12, 2019 · 55 min
Software Businesses with Hiten Shah
The software market changes every year. As individuals and enterprises become more willing to buy software, there are new markets for entrepreneurs to sell software to. Good software has high margins and high retention, so even a niche software business can prove profitable. As software spreads across the world, developing countries are showing a willingness to buy the same software tools as the developed countries, making the niche software businesses even more successful than expected. It’s a great time to start a software business. But the fundamental challenges faced by an entrepreneur have not gone away. An entrepreneur must find the necessary capital, build an initial product, iterate on that product, market to their audience, sell to their customers, and retain their users. Hiten Shah is an entrepreneur and investor who has started several businesses, both successful and unsuccessful. He is also the host of the Startup Chat podcast, a show about strategies and tactics for software businesses. Hiten joins the show to discuss his experience in the software industry, including lessons on financing, product development, and the future of software products.
#1160
Jun 11, 2019 · 1 hr 7 min
Kubernetes Vision with Joe Beda
Google Cloud was started with a vision of providing Google infrastructure to the masses. In 2008, it was not obvious that Google should become a cloud provider. Amazon Web Services was finding success among startups who needed on-demand infrastructure, but the traditional enterprise market was not yet ready to buy cloud resources. Googlers liked the idea of becoming a cloud provider. But was it the right time to enter the market? Google’s advertising business was a large and growing cash cow. Executives within Google were not sure how much capital and effort should be allocated into an infrastructure business. When Google decided to go into the cloud business, Joe Beda was one of the engineers who helped lead the effort, and joins the show as today’s guest. Google’s internal server infrastructure is managed by Borg, a system for allocating resources to applications. Google Cloud runs on Borg, and there were a number of early engineering challenges to building the necessary functionality into Borg for running a cloud provider on top of it. One example of a technical challenge that Google faced was the refactoring of Borg to run Google Cloud workloads. The requirements for public infrastructure are different than those of internal Googlers. Inside of Google, developers deploy their applications to containers running on bare metal. Outside of Google, developers want to create virtual machines. Borg needed to be refactored in order to instantiate VMs. Google solved this technical problem, as well as many other challenges, and Google Cloud slowly gained momentum in the market. But AWS remained the default choice for profitable enterprise workloads. It wasn’t until the container orchestration wars that Google found an opportunity to jump on a market segment that offered strong differentiation. By open sourcing Kubernetes and presenting a clear vision for where the project was going, Google shifted the battlefield of the public cloud toward a competitive landscape where it has many advantages. Kubernetes also provided many other technology companies with an opportunity to get into the cloud market, creating a collaborative, multi-company ecosystem that has accelerated the pace of software faster than anyone expected. Joe Beda has been instrumental in the evolution of the cloud native ecosystem. In today’s episode, Joe gives his memories on Google Cloud, Kubernetes, and his Kubernetes company Heptio, which he sold to VMware.
#1159
Jun 10, 2019 · 55 min
Kubernetes Storage with Saad Ali
Containers are made to fail gracefully. When your container shuts down due to a hardware or software failure, your distributed application should be able to tolerate that failure. One simple way to be able to tolerate such a failure is to make all of your application logic “stateless.” If your application does not maintain state, then shutting it down in the middle of a computation is not a problem–you can just restart the application, restart your computation, and get the same result. But applications need to maintain state. We need to use databases and in-memory systems to manage long-lived user sessions and other interactions. A database is not just an on-disk abstraction–a database requires an application server to be accepting network traffic. We can run those database applications within containers. There is a fundamental tension between stateful applications and the idea that containers are meant to tolerate failure gracefully. Saad Ali is an engineer at Google, and he returns to the show to discuss Kubernetes storage and state management. He gave a keynote at KubeCon EU, which I spoke to him about. New Software Daily app for iOS. You can become a paid subscriber for ad free episodes at softwareengineeringdaily.com/subscribe
#1158
Jun 7, 2019 · 1 hr 14 min
Kubernetes Market with Adam Glick
Amazon Web Services is the leading cloud provider by a large margin. Amazon established its lead by being first to market in 2006, with Google and Microsoft taking several years to catch up to the huge business opportunity of the cloud. Since 2008, Google Cloud has been working on cloud products for developers. It started with App Engine, which is widely used internally at Google, but has not had overwhelming public adoption. Over the last eleven years, Google has refined its understanding of how customers want to buy public cloud resources. Google Cloud products like Cloud Storage, Persistent Disk, and BigTable have given Google parity with many of the AWS public cloud offerings. Although Google has caught up to AWS in terms of products, the enterprise market has continued to choose AWS as its default. AWS is widely perceived as having more experience in running enterprise workloads, and a better responsiveness to customers. In order to keep Amazon from running away with the cloud market entirely, Google needed to shift the competitive landscape to different territory. Kubernetes provided the paradigm shift that Google needed. The market for cloud providers has changed completely due to Kubernetes. When Google open sourced Kubernetes, it created a common codebase for software companies to build software for managing distributed systems. In the span of five years, Kubernetes has turned the world of cloud products into a world resembling the open source Linux ecosystem. This is a remarkable shift, and every infrastructure software vendor is still figuring out its strategy for adapting. Adam Glick is the head of modern infrastructure and serverless marketing at Google. With Craig Box, he hosts the Kubernetes Podcast from Google, an excellent show about recent developments and evergreen concepts within the world of Kubernetes. Prior to Google, Adam worked at AWS for 3 years and Microsoft for twelve years. He has seen each of the major cloud providers up close and has a deep awareness for how each company thinks. We had a great conversation about the cloud native landscape, podcasting, and developer marketing. New Software Daily app for iOS. You can become a paid subscriber for ad free episodes at softwareengineeringdaily.com/subscribe
#1157
Jun 6, 2019 · 1 hr
Service Mesh Interface with Lachlan Evenson
Containers offer a lightweight abstraction for running a server. Cloud providers are able to manage billions of containers from different users, allowing for economies of scale so that each user can pay less. Today, there is a variety of ways that users can deploy containers on a cloud provider. These containers can run in managed Kubernetes clusters, in functions-as-a-service, or in long-lived standalone container instances. User preferences are getting more sophisticated, with some users showing an interest in Knative, an open source serverless system originally created at Google. Whichever container deployment system you choose, your application and its multiple servers need a way to route traffic, measure telemetry, and configure security policy. A service mesh abstraction can help serve these use cases. Lachlan Evenson has worked in containers and Kubernetes since before the container orchestration wars. He was an engineer at Deis, a company which built an open source platform-as-a-service running on top of containers and Kubernetes. Deis was acquired by Microsoft, where Lachlan now works as principal program manager of Container Compute. Lachlan joins the show to discuss containers, Kubernetes, and the Service Mesh Interface, an interoperable service mesh layer that Microsoft launched with Buoyant.
#1156
Jun 5, 2019 · 1 hr 6 min
Multicloud Future with Bassam Tabbara
Each cloud provider offers a different set of services which are not always compatible with each other. What are the challenges of building an application that interoperates with multiple different clouds? The first issue is API compatibility. Most cloud providers have a managed SQL offering, a bucket storage system, and server abstractions like virtual machines and containers. But these tools might have different APIs on each cloud. The code that you wrote to save your application data to Amazon S3 might have to be rewritten if you decide to switch your bucket storage to a different provider. Another issue that interferes with cloud interoperability is the degree of integration on a particular cloud. If I build my application for AWS, I might be heavily integrated with Amazon’s Identity and Access Management policy system and AWS logging. Each cloud provider makes it particularly easy to connect to their integrated solutions. There is also the problem of services on one cloud that simply do not map to a service on any other cloud. Google Cloud Bigtable does not have an equivalent service on Amazon. Microsoft CosmosDB does not have an equivalent service on Digital Ocean. As developers, this irritates us. We want to be able to deploy our application to any cloud. We want to be able to move applications easily from one cloud to another. And we want to mix tools from different clouds together as easily as we import a library. Bassam Tabbara is the CEO of Upbound, a company focused on making multicloud applications easier to deploy and operate. I spoke to Bassam at KubeCon EU 2019, and he described the problems of multicloud deployments and the opportunities for the cloud native ecosystem to become more cross-compatible.
#1155
Jun 4, 2019 · 46 min
Kubernetes Development with Tim Hockin
Kubernetes has evolved from a nascent project within Google to a thriving ecosystem of cloud providers, open source projects, and engineers. Tim Hockin is a principal software engineer who has been with Google for 15 years. Tim joins the show to talk about the early days of the Kubernetes projects, and the engineering efforts that are under way five years into the project. At KubeCon EU 2019, two of the prevalent subjects of discussion were service mesh and serverless–particularly the Knative project. Tim gave his perspective for how projects that are adjacent to Kubernetes are developed within the community.
#1154
Jun 3, 2019 · 52 min
Google Anthos with Aparna Sinha
Google’s cloud business was long regarded as a place where startups could build a business, but not established enterprises. For serious workloads, enterprises chose Amazon almost unanimously. This phenomenon of Amazon as the default was described by a phrase that harkened back to the days of IBM’s dominance: “nobody ever got fired for choosing AWS.” With the rise of Kubernetes, Google has established a reputation as a reliable provider of container orchestration. As enterprises look to roll out Kubernetes workloads, Google is convincing many of them to work with Google Kubernetes Engine. Kubernetes is one part of the changes which fall under the description of “digital transformation”. As these enterprises build out their digital transformation strategy, they are also looking for a strategy around multicloud, on-prem, policy management, and consulting partnerships. Anthos is a platform where enterprises manage the resources and configuration of their cloud deployments, as well as a system to partner with services integrators and independent software vendors. Aparna Sinha works on Anthos, Google Kubernetes Engine, and other projects at Google. In today’s show we discuss the process of digital transformation, as well as the tactics for how a digital transformation is actually implemented. We also talked about GKE on-prem, and the kinds of tooling needed by on-prem application deployments.

Showing 1,181–1,200 of 2,221 episodes