Created a router configuration generator for multicast networks. As with most trading firms, Jane Street makes extensive use of multicast networking. Each time a new multicast network gets set up (e.g. building up a new data center) Network Engineers need to plan out the basic layout of the network, and then configure each router to know about the various groups, how to route messages, etc. This second step is a lot of work, and often requires duplicating similar information between different routers. And repeated work also means that there's potential to make mistakes each time the information is added to a new router. To improve this process, I spent the first half of my internship extending the internal networking library to parse, format, and generate these multicast configs. The final step of the project was setting up a tool that network engineers could use to translate their high-level description of a new data center into the individual configuration files for each router.

Added smarter caching to data-fetching framework. The second half of my internship was spent on the Commodities trading desk, where my main project consisted of improving caching in a new framework called Datafetcher. Datafetcher is essentially a fancy dependency injection framework, in that it provides the usual dependency injection features (easy testing) but can also automatically batch fetches and cache their results. A few internal apps had already been converted to using Datafetcher, and while it showed some instant performance improvements, there were still some situations where caching wasn't good enough. In particular, sometimes apps would fetch the same data multiple times and then run some further computations on the fetched data. While the initial fetches were all cached correctly, programs still had to repeat all the subsequent work because the final results of computations weren't cached. Since the computations all used deferred evaluation, it wasn't possible to simply memoize the results with existing tools. I spent a few weeks coming up with an interface for memoizing these computations (with the goal of making it type-safe and easy-to-use), then converted an existing app to use Datafetcher and spent some time documenting best practices for the framework.

Built a high-throughput end-to-end load testing framework. To gain confidence in mission-critical systems, it's important to know their performance charateristics and bottlenecks. Stress testing a system is a powerful way to measure its limitations and gain insight about its weakest links. During my internship, I was tasked with designing such a system for a new app. I later used this framework to find, document, and suggest improvements for several bottlenecks in the system. Before leaving, I also got to set this up as a regression test, so that we could be confident that performance would only improve with future changes.

Developed a unified distributed storage monitor. As a quantitative hedge fund, Two Sigma makes extensive use of big data. Hundreds of petabytes of data are stored in various systems across the firm. To help users understand the impact of their data, I designed a service that collected stats on user's storage locations. This came together as a REST API that allowed for querying these stats, giving users insight into the cost, impact, and size of their storage.

Researched solutions for porting native libraries to Node.js WebAssembly. WebAssembly promises a new model for writing web apps: high performance, cross-platform, secure, and most of all an easy compilation target. Node.js apps frequently interface with native libraries, but support for portable native libraries is lacking. Using feedback from Node.js users, I experimented with several ways to compile native libraries to WebAssembly and package them for use with Node.js. My work began with a simple C-to-Node.js bindings generation tool and ultimately culminated in a proof-of-concept port of the Node.js API (NAPI) to WebAssembly.

Built an education technology Software-as-a-Service startup from the ground up with two partners. Our product targetted work-based-learning students — high school students who engage in internships for academic credit. To help solve the problems of educators around the state, we designed an all-in-one platform to manage attendance, employee evaluations, messaging, and more.