NewsTech News

Software Engineering Daily

softwareengineeringdaily.com

Software Daily: The world through the lens of software.

Play

Open in

2,221 episodes
Avg 53 min
English

Episodes 2,221

Newest Oldest Longest

All20 Unplayed20 In progress0 Played0

Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.

#1015
Dec 14, 2018 · 56 min
Full Stack JavaScript with Wes Bos
Wes Bos has created popular courses on React, GraphQL, and JavaScript. With hundreds of thousands of students, Wes has earned a cult following for his fun, practical lessons on web development. The courses produced by Wes teach developers how to build useful applications such as a complete e-commerce store. Wes has built a career around studying and evangelizing JavaScript. His approach to education centers around practice, repetition, and hacking on fun projects. He also co-hosts a podcast called SyntaxFM, and is a frequent Twitter user. Wes is a rare mix of developer, teacher, businessman, and designer. Throughout his work, there is an artist’s sense of attention to detail, and a modern entrepreneur’s sense of pricing and marketing. His sites, such as JavaScript30 and React For Beginners have the deliberate style of someone who has been building websites for a very, very long time.In today’s episode, Wes Bos joins the show to give his perspective on JavaScript, entrepreneurship, and podcasting. To learn more about Wes’s business and his background, check out the IndieHackers podcast with him.
#1013
Dec 13, 2018 · 57 min
Plaid: Banking API Platform with Jean-Denis Greze
A bank account is a platform for apps to be built on top of. If that sounds like a weird idea, think about the features of a bank account. Most users only have a single bank account, making it a tool for identity and authentication. The series of transactions in a bank account provides a data set that can be used for analyzing payment history and issuing loans, or insurance. But there are difficulties to building a platform on top of banking. There are thousands of different banks. If you want to build an application that integrates with a user’s bank, you need to be able to integrate with any bank that the user might use–whether it’s Bank of America, Wells Fargo, or Chase. Plaid is a company that builds APIs for users to connect to banks. Applications such as Venmo, Betterment, and Coinbase use Plaid to connect with the bank accounts of their users. Jean-Denis Greze joins the show to explain how applications use Plaid, and how Plaid has scaled its infrastructure to handle a high volume of requests. He also discusses the potential of banking as a platform, and the strategy for expanding the APIs that Plaid can offer to developers. Fintech Daily is a new podcast from Software Engineering Daily covering payments, cryptocurrencies, trading, and the intersection of finance and technology. We are looking for volunteer hosts for Fintech Daily, and if you are interested in working with us to conduct interviews, send an email to host@fintechdaily.co. You can find the podcast on iTunes, Google, and everywhere else, and if you are interested in hosting, don’t hesitate to reach out.
#1011
Dec 12, 2018 · 52 min
High Growth Handbook with Elad Gil
When a startup finds product market fit, the adoption of that product can grow rapidly, turning a startup into a high growth company. All of a sudden, a startup that was struggling to find its first customer is bombarded with new challenges. The startup has to hire tens of new employees. This requires raising capital, so the startup has to meet with investors and lawyers. A rapid influx of new customers puts a strain on the engineering and customer service elements of the company. There is too much to do, and there is only so much time in a day. The CEO of the high-growth company is up late into the night, answering emails and losing sleep. But these are good problems to have, and the company is in a state of exuberance. The CEO must balance psychological health with the stressful task of scaling a company. Elad Gil is an entrepreneur and author of “High Growth Handbook”, a book of lessons and guidelines about how to navigate a startup that has found product market fit, and is beginning to scale. High Growth Handbook includes interviews with experienced entrepreneurs such as Marc Andreessen and Patrick Collison, whom Elad met with as he wrote the book. Elad joins the show to discuss his book, and his own personal lessons of working with companies such as Twitter, Google, Stripe, and Coinbase. Elad has worked at several high growth companies and invested in others, and he has gathered a lot of wisdom from these different experiences.
#1009
Dec 11, 2018 · 33 min
Feature Flags with Edith Harbaugh
Releasing software has inherent risk. If your users don’t like your new feature, they might stop using your product immediately. If a software bug makes it into production, it can crash your entire application. Releasing software gradually has many benefits. A slow rollout to an increasing population of users allows you to test your software in multiple real-world environments before it goes live to everyone. A system of AB testing different versions of your software lets you see how different flavors of your software perform against similar audiences. Edith Harbaugh is the CEO of LaunchDarkly, a system for feature management. LaunchDarkly allows developers to deploy new software releases in a controlled fashion. Edith joins the show to discuss how to implement feature flagging, and why an intelligent release process can lead to a more scientific, predictable environment for software development. Edith is also the host of To Be Continuous, a podcast about continuous delivery, software engineering, and DevOps.
#1007
Dec 10, 2018 · 1 hr 7 min
Serverless Research with Ion Stoica
The Berkeley AMPLab was a research lab where Apache Spark and Apache Mesos were both created. In the last five years, the Mesos and Spark projects have changed the way infrastructure is managed and improved the tools for data science. Because of its proximity to Silicon Valley, Berkeley has become a university where fundamental research is blended with a sense of industry applications. Students and professors move between business and academia, finding problems in industry and bringing them into the lab where they can be studied without the day-to-day pressures of a corporation. This makes Berkeley the perfect place for research around “serverless”. Serverless computing abstracts away the notion of a server, allowing developers to work at a higher level and be less concerned about the problems inherent in servers–such as failing instances and unpredictable network connections. With serverless functions-as-a-service, the cloud provider makes guarantees around the execution of serverless code–such as with AWS Lambda. With serverless backend services, the cloud provider makes guarantees around the reliability of a database or queueing system. The cloud provider is operating servers to power this functionality. But the user is not exposed to those servers. Today’s show centers around the serverless functions-as-a-service. This is a new paradigm of computing, and there are many open questions. How can the servers for our functions be quickly provisioned? How can we parallelize batch jobs into functions as a service? How can large numbers of serverless functions communicate with each other reliably to coordinate? In production applications, functions-as-a-service are mostly used for “event-driven” applications. But the potential for functions-as-a-service is much larger. Ion Stoica is a professor of computer science at Berkeley, where he leads the RISELab. He is the co-founder of Conviva Networks and Databricks. Databricks is the company that was born as a result of the research on Apache Spark. Ion now serves as executive chairman of Databricks. Ion joins the show to describe why serverless computing is exciting, the open research problems, and the solutions that researchers at the RISELab are exploring.
#1004
Dec 7, 2018 · 57 min
Technical Investing with Sunil Dhaliwal
Robotics, genomics, and backend infrastructure: as an in vestor, it can be difficult to assess the viability of a startup that is on the cutting edge in any of these areas. A robotics startup requires a team with an integrated understanding of hardware and software. A genomics company will not only have to develop a successful healthcare product, but will have to bring it to market through regulation. And in the world of backend infrastructure, building a business that will be differentiated from giant cloud providers gets harder every day. Amplify Partners is a venture capital fund with an emphasis on technical investments. Their portfolio includes infrastructure companies like Datadog and Gremlin, as well as pharmaceutical and hardware companies. Sunil Dhaliwal is the founder of Amplify Partners, and joins the show to discuss the thesis of Amplify. The investments that Amplify makes are in technical companies–which makes these financing decisions complex enough to require detailed, individualized research. But there are commonalities among the founding teams. Sunil lays out a useful rubric for anyone who is looking to learn about venture capital investing.
#1002
Dec 6, 2018 · 54 min
RapidAPI: API Marketplace with Iddo Gino
Building software was simplified by cloud providers. With the cloud, it became much easier to deploy a server, spin up a database, and scale an application. Cloud providers like AWS gave developers access to these infrastructure primitives like storage and compute. On top of those primitives, numerous API companies have been built. An API company offers a more specific set of services. Twilio offers SMS text messaging API services. Stripe offers payment API services. These APIs give developers another level of tooling to build software out of. Developers can now think of entire applications in terms of APIs, and the number of APIs is growing rapidly. From business services such as booking a flight to machine learning models like image classification, the “API economy” has given developers a huge catalog of tools. Since developers have this additional leverage, software can be built with smaller teams. The codebase can also be smaller. But one area where the complexity is growing is the number of APIs that need to be managed. For each API, there is a different system for integrating the API into your application. Different API providers have different levels of reliability. Another area of difficulty is the discoverability of APIs. If I don’t know about a flight search API, I am never going to think of what applications I could build on top of that. There are APIs for generating memes, and APIs for easily querying what music is trending across the world. RapidAPI is a marketplace for APIs. It includes search and discovery features for the wide variety of different APIs that can be found across the internet. RapidAPI is also a system for integrating with multiple APIs through it’s API management system. Iddo Gino is the CEO and founder of RapidAPI, and he joins the show to discuss the motivation for creating an API marketplace, as well as the engineering behind RapidAPI.
#1001
Dec 5, 2018 · 55 min
Bitcoin Payment Channels with Alex Bosworth
The Bitcoin main chain is a large distributed ledger of transactions. Bitcoin is useful for maintaining a trusted record of payments, but is not practical for small day-to-day payments. Bitcoin payment channels allow users to issue small payments to each other without paying the high transaction cost and latency of going through the main chain. When payment channels are connected to each other, a “lightning network” is formed. Lightning network is often referred to as a “second layer” scalability solution. Alex Bosworth is a lightning infrastructure lead at Lightning Labs, a company that builds infrastructure for scaling blockchains. In today’s show, Alex explains how Bitcoin payment channels work, and provides some context on how developed the modern infrastructure is in terms of practical use cases for Bitcoin.
#999
Dec 4, 2018 · 47 min
Streaming Platform Architecture with Luca Mezzarlira and Yan Cui
Demand for live streaming video over the internet is increasing. After the emergence of early live streaming platforms, like Twitch and Facebook Live, more forms of video have become accessible over live streams, such as sports. Live streaming is a harder engineering problem than delivering a static video file because the information distributed on a live stream is constantly changing. DZN, spelled D-Z-N, is a live streaming service for watching fight events, such as boxing. The workloads for live streaming can be highly bursty. When a fight is scheduled to happen, the vast majority of traffic will hop on to watch the fight 20 seconds before the fight starts. A huge number of users logs into DZN and starts watching all at the same time. This quick spike in traffic means that DZN has to have servers spun up and be ready in advance. Luca Mezzarlira and Yan Cui are engineers at DZN. Yan was previously on the show in a few amazing episodes to talk about serverless infrastructure and the complexities of real-time video game software development. Those episode links are in the show notes. I highly recommend checking them out. Today’s show is a discussion of architecting a system to handle a high bandwidth customer use case. I hope you like it.
#997
Dec 3, 2018 · 57 min
On-Prem Cloud with Bob Fraser
Not every company wants to move to the public cloud. Some companies have already built data centers, and can continue to operate their business with their own servers. Some companies have compliance issues with the public cloud, and want to operate their own servers to avoid legal risk. Operating a data center is not easy. Operating systems need to be updated and security vulnerabilities need to be patched. Servers fail, and their workloads need to be automatically scheduled onto other servers to avoid downtime. In contrast to classic on-prem data center management, the cloud provides many benefits: automatic updates, an infinite pool of resources, fully programmable infrastructure as code. In the cloud, developers can provision infrastructure with an API request. Continuous delivery pipelines can be spun up at the click of a button. This tooling makes it dramatically easier for developers to move quickly, and for a business to move faster. Companies that operate their own data center want to be able to have these benefits of the cloud while still controlling their own infrastructure. Today’s guest Bob Fraser works at HPE on OneView, a tool for managing on-prem infrastructure like a cloud. Bob describes the difficulties of managing legacy on-prem infrastructure, and the advantage of building a management layer on top of data center infrastructure to make it more programmable. We’ve done lots of shows recently about Kubernetes in the context of cloud computing. Today’s show outlines how modern on-prem infrastructure can be managed like a cloud. Full disclosure: HPE is a sponsor of Software Engineering Daily.
#995
Nov 30, 2018 · 52 min
Anchor: Podcast Platform with Nir Zicherman
Podcasts have surged in popularity, but the podcast ecosystem remains difficult to work with. Podcast listeners have difficulty finding episodes. Podcast creators have difficulty finding out how to get started. The advertising marketplaces for podcasts are immature, and it can be difficult to build a business as a podcaster. Podcasting is unlike almost any other media format that we consume on the Internet. There is not an algorithmic feed of podcasts–we subscribe to podcasts we like and we see everything that gets published. Podcasting originated with Apple, who has not shown much interest in the podcast medium. Anchor is a platform that makes it easy for users to publish podcasts. Today, a large percentage of the new podcasts created on the Internet are started on Anchor. Nir Zicherman is the CTO at Anchor, and he joins the show to discuss the strange world of podcast technology, and how Anchor is building a business.
#993
Nov 29, 2018 · 53 min
Cloud Costs with Ran Rothschild
Cloud computing changed the economics of running a software company. Before the cloud, a software company had to purchase physical machines which often required thousands of dollars paid up front. The cloud allowed developers to deploy their applications for free, to operate a business for cheap, and to scale without hiring a dedicated team to manage the servers. Building in the cloud is cheap, but scaling in the cloud can get expensive. A growing company can often save money by changing which cloud instances and services they use. Reducing the number of server instances, changing the size of compute instances, and changing rules around auto scaling. By using monitoring, dashboards, and regular analysis of where money is spent, a business can find thousands of dollars of wasted spend per month. There are also broad strategic decisions around cost. One area to study is the use of “managed” services like Amazon DynamoDB, Google BigQuery, and Amazon Lambda. These services are proprietary, and can lead to lock-in. Sometimes they can be quite expensive. But they can save developers hours of time because they are easy to use, and provide high uptime guarantees. Ran Rothschild works at DoIT International, a company that helps businesses figure out how to save money on their cloud infrastructure. He joins the show to discuss the places where the most money is wasted and how startups can manage their infrastructure in a cost-effective manner. He also tells some stories about significant overspend. Full disclosure: DoIT International is a sponsor of Software Engineering Daily.
#991
Nov 28, 2018 · 57 min
Slack Messaging Architecture with Keith Adams
Slack is a real-time messaging system for work communication. On Slack, chat rooms as big as 100,000 people have productive conversations. This might sound like the same problem solved by social networks like Facebook, where billions of users communicate over a newsfeed. But the engineering constraints of a messaging system are different than that of a social network. On a newsfeed, the order in which events appear is not chronological. Events can be out of order. You can miss events. When a user posts a message to a social network, there are not strict guarantees around when other people will see that message. On Slack, messages have strong guarantees around arrival. When I send a message, everyone else who is in the room and connected should quickly receive that message as well. The messages need to be ordered and delivered exactly once. All messages on Slack are persisted. We have covered the architecture and security model of Slack in previous shows. In today’s show, Keith Adams returns to discuss how messages are processed and broadcast in Slack. The problem of Slack’s messaging system is similar to the distributed systems problem of “atomic broadcast”, in which a single process broadcasts a message which needs to be received by all other processes correctly–or else received by none of them. In Keith’s last show, he talked through the benefits of building a large system on PHP. He worked on infrastructure at Facebook, which was also a PHP application. It’s worth noting that both Slack and Facebook have scaled a monolithic architecture.
#989
Nov 27, 2018 · 56 min
Facet Wealth Engineering with Gorkem Sevinc
Many people have saved some money which they want to invest for the future. Some people are happy investing their money in a roboadviser, which programmatically puts money into long-term investments. Other people want a more personal approach involving a certified financial planner (CFP). A CFP is a human who allocates capital for an individual based off of that individual’s preferences. A CFP spends time and effort researching the options for a client. If the client only has a small amount of money (say, $15,000), it is not worth it for the CFP to spend much time on that account. As a result, there is a type of client who has saved a little bit of money, but has not saved enough to be an important client for a CFP. Facet Wealth is a software company that makes software for CFPs to work more effectively with their client accounts. Facet Wealth has in-house CFPs who work with this software to manage client accounts. In addition, Facet Wealth buys client accounts from independent CFPs who have small accounts which they do not have time to manage. This is an innovative way to aggregate users onto the platform. Gorkem Sevinc is CTO at Facet Wealth. He joins the show to describe the business and the software architecture of the company. We touched on many different areas–from human-computer interaction to the future of investing. We recently launched a new podcast: Fintech Daily! Fintech Daily is about payments, cryptocurrencies, trading, and the intersection between finance and technology. You can find it on fintechdaily.co or Apple and Google podcasts. We are looking for other hosts who want to participate. If you are interested in becoming a host, send us an email: host@fintechdaily.co
#986
Nov 26, 2018 · 50 min
Parity: Blockchain Infrastructure with Gavin Wood
Parity is a company that builds blockchain infrastructure. Parity has built several open source projects and works with enterprises to put blockchain technology in production. Gavin Wood is the founder of Parity, and he joins the show to talk about the state of blockchain technology and what his company is currently focused on. Four years ago, Gavin helped start the Ethereum project, so he has lots of context on decentralized technology. Gavin envisions a world with many different blockchains for many different use cases. These blockchains will interact with each other to enable trusted relationships between parties. One project that Parity has created is Substrate, a technology that allows developers to quickly stand up a blockchain with the right privacy level. Another project is Polkadot, which allows blockchains to connect and interoperate with each other. Gavin and I discussed why the world needs a variety of blockchains–and whether all of these different blockchains should need their own cryptocurrency. Gavin described the use case of blockchains for mediating supply chain trust. We also talked about the technologies used to build these projects, including WebAssembly and Rust.
#985
Nov 23, 2018 · 1 hr 2 min
Death and Distributed Systems with Pieter Hintjens Holiday Repeat
Originally posted on June 23, 2016. Pieter Hintjens grew up writing software by himself. The act of writing code brought him great pleasure, but the isolated creative process disconnected him from the rest of the world. As his life progressed he became involved in open source communities, and he discovered a passion for human interaction. Open source software succeeds or fails on the strength of the community. One story of success is ZeroMQ, a popular open source distributed messaging system that was started by Pieter Hintjens. In this episode, Pieter gives his thoughts on human nature, distributed systems, and death. “A Protocol For Dying” is a blog post Pieter wrote recently, where he discussed his terminal diagnosis of cancer, and how it has reframed his perspective on life.
#984
Nov 22, 2018 · 56 min
Algorithms to Live By with Brian Christian Holiday Repeat
Originally posted on May 12, 2016. When you are deciding who to marry, you are using an algorithm. The same is true when you are looking for a parking space, playing a game of poker, or deciding whether or not to organize your closet. Algorithms To Live By is a book about the computer science of human decisions. It offers strategies for how to think through everyday life like a computer scientist. Brian Christian has a background in computer science and philosophy, and is an author of Algorithms to Live By. He joins the show to explain how the same algorithms and data structures we use for our computer programs can be applied to the real world.
#983
Nov 21, 2018 · 48 min
Poker Artificial Intelligence with Noam Brown Holiday Repeat
Originally posted on May 12, 2015. Humans have now been defeated by computers at heads up no-limit holdem poker. Some people thought this wouldn’t be possible. Sure, we can teach a computer to beat a human at Go or Chess. Those games have a smaller decision space. There is no hidden information. There is no bluffing. Poker must be different! It is too human to be automated. The game space of poker is different than that of Go. It has 10^160 different situations–which is more than the number of atoms in the universe. And the game space keeps getting bigger as the stack sizes of the two competitors gets bigger. But it is still possible for a computer to beat a human at calculating game theory optimal decisions–if you approach the problem correctly. Libratus was developed by CMU professor Tuomas Sandholm, along with my guest today Noam Brown. The Libratus team taught their AI the rules of poker, they gave it a reward function (to win as much money as possible), and they told it to optimize that reward function. Then they had Libratus train itself with simulations. After enough training, Libratus was ready to crush human competitors, which it did in hilarious, entertaining fashion. There is a video from Engadget on YouTube about the AI competing against professional humans. In this episode, Noam Brown explains how they built Libratus, what it means for poker players, and what the implications are for humanity–if we can automate poker, what can’t we automate? Stay tuned at the end of this episode for the Indeed Prime tip on hiring developers.
#982
Nov 20, 2018 · 1 hr 34 min
Salary Negotiation with Haseeb Qureshi Holiday Repeat
Featured Image Photo Credits Originally posted on July 11, 2016. Negotiation is an important skill for software engineers. The salary you negotiate at the beginning of your job could be a difference of tens of thousands of dollars over the course of an engineer’s career, but intimidating recruiters and exploding offers scare many engineers from negotiating at all. Today, Haseeb Qureshi returns to the show to discuss his epic story of salary negotiation. On a previous episode, Haseeb discussed leaving his career as a poker player to join a coding boot camp and start down the path of a software engineer. In this episode and his recent blog post, Haseeb describes his approach to the job search and salary negotiation process, which eventually landed him at Airbnb with a $250,000 annual salary–after about a year of learning to code.
#980
Nov 19, 2018 · 57 min
Schedulers with Adrian Cockcroft Holiday Repeat
Originally published on July 6, 2016. Scheduling is the method by which work is assigned to resources to complete that work. At the operating system level, this can mean scheduling of threads and processes. At the data center level, this can mean scheduling Hadoop jobs or other workflows that require the orchestration of a network of computers. Adrian Cockcroft worked on scheduling at Sun Microsystems, eBay, and Netflix. In each of these environments, the nature of what was being scheduled was different, but the goals of the scheduling algorithms were analogous–throughput, response time, and cache affinity are relevant in different ways at each layer of the stack. Adrian is well-known for helping bring Netflix onto Amazon Web Services, and I recommend watching the numerous YouTube videos of Adrian talking about that transformation.

Showing 1,321–1,340 of 2,221 episodes