NewsTech News

Software Engineering Daily

softwareengineeringdaily.com

Software Daily: The world through the lens of software.

Play

Open in

2,221 episodes
Avg 53 min
English

Episodes 2,221

Newest Oldest Longest

All20 Unplayed20 In progress0 Played0

Counted on this page — what you have heard stays on this device, so it is not something the list can be paged by.

#452
Jan 16, 2017 · 46 min
Python Data Visualization with Jake VanderPlas
Data visualization tools are required to translate the findings of data scientists into charts, graphs, and pictures. Understanding how to utilize these tools and display data is necessary for a data scientist to communicate with people in other domains. In this episode, Srini Kadamati hosts a discussion with Jake VanderPlas about the Python ecosystem for data science and the different attempts at creating a data visualization library. Jake VanderPlas is the Director of Research for Physical Sciences at the University of Washington’s eScience institute, where he also received his PhD in Astronomy. In addition to contributing to many Python data science libraries like scikit-learn, scipy, numpy, and matplotlib, he’s written multiple books that have been published by O’Reilly and has given many talks on data science tools and techniques. He’s also the co-creator of the Altair project, which is a declarative data visualization library for Python built on the Vega-Lite visualization grammar.
#451
Jan 13, 2017 · 1 hr
Columnar Data: Apache Arrow and Parquet with Julien Le Dem and Jacques Nadeau
Column-oriented data storage allows us to access all of the entries in a database column quickly and efficiently. Columnar storage formats are mostly relevant today for performing large analytics jobs. For example, if you are a bank, and you want to get the sum of all of the financial transactions that took place on your system in the last week, you don’t want to iterate through every row in a database of transactions. It is more efficient to just look at the column for the amount of money, and ignore things like timestamp and user id. Julien Le Dem co-created Parquet, a file format for storing columnar data on disk. Jacques Nadeau is a VP of Apache Arrow, a format for in-memory columnar representation. They are both part of Dremio, and they join the show to talk about how columnar data is stored, processed, and shared between systems like Spark, Hadoop, and Python.
#450
Jan 12, 2017 · 1 hr 5 min
Architecture of Free Code Camp with Berkeley Martinez
Free Code Camp is an online learning platform that takes people from knowing nothing about code to having enough knowledge to build software for a living. We have already done a show with Quincy Larson, the founder of Free Code Camp, in which we discussed his motivation for starting the organization. The economics of running a free interactive platform with thousands of users informs the architectural strategy. Free Code Camp does not even have ads, so it needs to be frugal. Berkeley Martinez is the CTO of Free Code Camp, and in today’s episode he discusses how Free Code Camp works from a technical perspective, including how the site’s sandboxed coding environment is built.
#449
Jan 11, 2017 · 49 min
Ethics of Software with Bill Sourour
Software engineers are often one of the last lines of defense against potentially dangerous and unethical practices. Every software company encounters situations where the line between right and wrong is not clearly drawn. Back in the year 2000 Bill Sourour was asked to code a project to help market a pharmaceutical product in a misleading way. Reflecting on this experience, Bill recently wrote about it on Medium, suggesting that developers should avoid doing things that compromise our ethics in small ways. The more software defines our real world the more relevant the conversation around ethics becomes. In this episode, Bill and I explore the ethics of modern software development. The Code I’m Still Ashamed Of
#448
Jan 10, 2017 · 59 min
Email Infrastructure with Chris McFadden
A company like Pinterest has millions of transactional emails to send to people. The scalability challenges of sending high volumes of email mean that it makes more sense for most companies to use an email as a service product rather than building their own. Chris McFadden is the VP of engineering and cloud operations at SparkPost and he joins the show to explain the architecture of SparkPost’s email as a service product. SparkPost started as an on-premise email technology for large enterprises, and evolved into a SaaS product. In 2014, the company migrated to the cloud, which has changed its infrastructure as well as its operational model. Full disclosure: SparkPost is a sponsor of Software Engineering Daily.
#447
Jan 9, 2017 · 58 min
Cyber Warfare with Jared Smith
Vulnerabilities exist in every computer system. As a system gets bigger, the number of vulnerabilities magnifies. The web is the biggest, most complex computer system we have–but fortunately, the steps we can take to secure our web applications are often quite simple. Jared Smith is a cyber security research scientist with Oak Ridge National Laboratory. He joined me on the show to discuss web application security, but I really wanted to know his position on some of the more grandiose topics–Stuxnet, our power grid, Russian hacking, and corporate backdoors. This was a wide ranging discussion and I enjoyed it a lot. For a presentation Jared gave at Nodevember about Web Security, check out this YouTube video.
#446
Jan 6, 2017 · 57 min
Meetup Architecture with Yvette Pasqua
Meetup is an online service that allows people to gather into groups and meet in person. Since 2002, the company has been growing and its technology stack has been changing. Today, they are in the process of migrating to the cloud, using both Amazon Web Services and Google Compute Platform. Yvette Pasqua is the CTO of Meetup and she joins the show to explain how Meetup’s technology stack works and how the teams are organized. The discussion of multiple clouds is particularly interesting–Yvette describes GCP and AWS as both having distinct, well-defined use cases at Meetup.
#445
Jan 5, 2017 · 55 min
Evolutionary Architecture with Neal Ford
When a useful new technology comes out, companies that are in a position to adopt that new technology can gain an edge over competitors. As our industry grows and moves faster, these kinds of changes are coming faster–some recent examples are Docker, ReactJS, and Kubernetes. Evolutionary architecture supports incremental, guided change as a first principle along multiple dimensions. A company with an evolutionary architecture is structured to evolve in response to changes inside the company (such as a decision to change the product) or outside the company (such as the emergence of Docker). Neal Ford is an architect at ThoughtWorks and one the creators of the evolutionary architecture concept.
#444
Jan 4, 2017 · 1 hr
Security Research with Samy Kamkar
Every digital system has vulnerabilities. Cars can be hacked, locked computers can be exploited, and credit cards can be spoofed. Security researchers make a career out of finding these types of vulnerabilities. Samy Kamkar’s approach to security research is not just about dissection–it’s also about creativity. For many of the technologies he hacks on, Samy open-sources code that summarily describes the vulnerability he has been working on. For example, in his project PoisonTap, Samy open-sourced code that you can run on a $5 Raspberry Pi, and plug into a locked computer to exploit it. Our conversation covered the art of deconstructing technologies for vulnerabilities and Samy’s goals as a security researcher. We also touched on some of the broader issues of modern security.
#443
Jan 3, 2017 · 57 min
Self-Contained Systems with Eberhard Wolff
Self-contained systems is an architectural approach that separates the functionality of a system into many independent systems. Each self-contained system is an autonomous web application, and is owned by one team. Communication with other self-contained systems or 3rd party systems is asynchronous where possible. As Eberhard Wolff explains in this episode, self-contained systems is not the same thing as microservices, but they are not mutually exclusive. Organizations often adopt a mix of architectural ideas, and it is worth understanding these different models so you can decide which of them to apply to your own projects.
#442
Jan 2, 2017 · 1 hr 19 min
Breaking Into Startups
Many people find themselves going down a career path that does not bring them satisfaction. A lawyer finds himself constantly working cases he doesn’t care about. A student in medical school gets completely burned out from hospital bureaucracy. An investment banker no longer finds joy in the accounting statements that she used to enthusiastically study. Startups offer a different career path. Within a startup, an employee can often find creativity, limited bureaucracy, and highly variable reward structures. These ingredients make the world of startups both refreshing and intimidating to someone who feels stuck in a career that no longer gives them joy. Breaking Into Startups is a podcast about people coming from non-traditional backgrounds and making their way into startups. Breaking Into Startups is run by Artur and Timur Meyster and Ruben Harris, who join me for a conversation about how they migrated from more traditional careers to startups–and how many other people are doing the same.
#441
Dec 29, 2016 · 1 hr 1 min
Startup Engineering with Mike Wolfe
In the 1990s, the barriers to starting a company were significant. Not only did you need an idea, you needed $200,000 for servers and Oracle licenses. With cloud computing, the up-front financial costs of getting a company off the ground have been mostly eliminated–but the idea of starting a company is still perceived as risky. The process of building software has changed dramatically in the last twenty years, but many of the challenges of managing a software company remain timeless. How do you hire properly? How do you keep track of a software stack that is growing in complexity? How do you handle dissenting opinions from employees? Mike Wolfe has been building software companies since the 1990s and joins the show to discuss how to build products and manage engineering teams. It’s a wide-ranging discussion including technological trends, interpersonal skills, and startup financing.
#440
Dec 28, 2016 · 55 min
Robot Cloud Lab with Max Hodak
A biologist wants to study the genetic makeup of an organism. A pharmaceutical researcher wants to test the effects of an experimental drug. These types of experiments require a deep knowledge of the scientific domain as well as the lab techniques to produce the data that will eventually yield a result. Transcriptic is a robotic biology laboratory that allows you to make requests for experiments to run remotely. Scientists who use Transcriptic do not have to perform wet lab experimentation. The robotic laboratory process has the potential to do for biology what Amazon Web Services did for computing. Max Hodak is a cofounder of Transcriptic and joins me today to explain how a robotic laboratory works. There are a wide variety of challenges, from hardware integration to software reverse engineering–but we also talk about why it is worth it to overcome these challenges, including a discussion of the “reproducibility crisis” that is undermining faith in our scientific experiments.
#439
Dec 27, 2016 · 59 min
Performance Monitoring with Andi Grabner
Application performance monitoring helps an engineer understand what is going on with an application. An application on a single machine is often monitored by inserting bytecode instructions into the application after it has been interpreted. Distributed cloud applications with functionality broken up across multiple servers often use distributed tracing. Andi Grabner from Dynatrace joins today’s show to explain how monitoring software is built, and how engineers use it to solve problems. Monitoring is core to every business, whether the goal is to understand top-level business processes of to dissect and debug a specific engineering problem. And because monitoring is important at every layer of the stack, there is a plethora of monitoring tools for sale.
#438
Dec 23, 2016 · 54 min
Antifraud Architecture with Josh Yudaken
Online marketplaces and social networks often have a trust and safety team. The trust and safety team helps protect the platform from scams, fraud, and malicious actors. To detect these bad actors at scale requires building a system that classifies every transaction on the platform as safe or potentially malicious. Since every social platform has to build something like this, Smyte decided to engineer trust and safety as a service. Josh Yudaken joins the show today to discuss how Smyte engineered its platform to provide machine learning models for any organization that wants to take advantage of Smyte for its trust and safety. The tools we discuss include Kubernetes, RocksDB, and Kafka, and Smyte is solving some problems that have not been solved before, so this is a great episode for anyone interested in data engineering or fraud detection–or how to use cloud services and open source tools in unique ways.
#437
Dec 22, 2016 · 52 min
Sampling with Daniel Trostli
Every song you hear on the radio is written with a computer. Computer musicians mostly use synthesizers and samples to compose these songs. A sample is a snippet of recorded sound, sometimes taken from a songs, a movie, or another source. The more samples a musician has access to the better. SampleFocus is a platform where musicians upload and download samples to build songs with. Daniel Trostli is the engineer who is building SampleFocus, and he joins the show to discuss how people use samples to write music and how he built SampleFocus. I met Daniel at the Launch Scale event, a conference about building businesses, and when I was having lunch with Daniel, he told me about working on this personal project full-time, and some of the different business models he was exploring. It’s a great episode for anyone who is thinking about turning a side project into a business.
#436
Dec 21, 2016 · 53 min
Data Engineering with Pete Soderling
In the last five years, companies started hiring data engineers. A data engineer creates the systems that manage and access the huge volumes of data that are accumulating on cheap cloud servers. As the saying goes, “it’s more expensive to throw out the data than to store it.” Pete Soderling joins the show to discuss the rise of the data engineer, and how that role interacts with data scientists, software engineers, and machine learning experts. Pete runs Hakka Labs, a site that aggregates material for data science and data engineering. He also organizes DataEngConf, a conference for data engineering.
#435
Dec 20, 2016 · 1 hr
Ad Fraud Research with Augustine Fou
A huge percentage of online advertisements are never seen by humans. They are viewed by bots–automated scripts that are opening web pages in a browser and pretending to be a human. Advertising scammers set up web pages, embed advertisements on those pages, and then pay for bot traffic to come and view those advertisements. This aspect of the internet is bizarre and alarming. Think about it–how much of the Internet economy depends on online advertising? A lot! How many of those advertisements are consumed by robots? Some estimates say as much as 80%. The more time I spend looking at the online advertising industry, the more perplexed and curious I become. Augustine Fou also has this curiosity. He got a PhD at MIT in material science and engineering–but today he spends his time researching advertising fraud and working as an independent consultant.
#434
Dec 19, 2016 · 58 min
Reactive Microservices with Jonas Boner
For many years, software companies have been breaking up their applications into individual services for the purpose of isolation and maintainability. In the early 2000s, we called this pattern “service-oriented architecture”. Today we call it “microservices”. Why did we change that terminology? Did the services get smaller? Not exactly. Jonas Boner suggests that the movement towards cloud and the increased prevalence of mobile changes how we look at these services–so much that we needed to change the terminology necessary to even talk about them. And once the conversation has shifted to “microservices”, what steps do we need to take to implement them properly? The reactive manifesto is a collection of principles for how to build applications. When the reactive manifesto is applied to the idea of microservices, we get reactive microservices, which Jonas and I discuss in today’s episode.
#433
Dec 16, 2016 · 48 min
Scale API with Lucy Guo and Alexandr Wang
Some tasks are simple, but cannot be performed by a computer. Audio transcription, image recognition, survey completion–these are simple procedures that almost any human could execute, but the machine learning models have not gotten consistent enough to do them accurately. Scale is an API for human labor, created by Lucy Guo and Alexandr Wang. Similar to Amazon Mechanical Turk, Scale sends small, simple tasks to workers who can complete those tasks. Scale provides an interface that is easy for developers to use, unlike Mechanical Turk, which requires a dashboard. Similar to how Stripe allows developers to build software off of payments systems easily, Scale allows developers to build human-driven, manual tasks into their code–which unlocks a wide range of potential applications, which Lucy and Alexandr discussed with me.

Showing 1,821–1,840 of 2,221 episodes