MidJourney-style image progress using SDXL, Diffusers, WebSockets, React, and FastAPI.

As the MidJourney Discord bot is generating images for your prompt over some number of diffusion steps, it makes the wait time a little more enjoyable by showing you your images at the current step. I was surprised that there wasn't any paid API or simple code on GitHub that had this feature, given that it's the best text2image user experience I've seen, so I built the frontend and backend using SDXL, Diffusers, WebSockets, React, and FastAPI and made the code open source.

Reconstructing the Mind’s Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors

This is a paper I wrote with Stability AI MedARC that uses contrastive learning and diffusion priors to essentially read minds. It decodes the image that a person is viewing while in an fMRI machine using only the measured brain activity and sets a new state-of-the-art result for this task. Accepted as a spotlight for the NeurIPS 2023 conference. This was a great experience with open science and really highlighted to me the power of this format.

JupyterSpot.com - collaborative whiteboards for Jupyter notebooks

I developed this website to transform Jupyter notebooks into collaborative whiteboards similar to Figma, with the goal of streamlining remote data science meetings and reducing the frequent, "Can you scroll up a bit? No, too far, go back down..." interruptions. It incorporates Figma-esque functionalities such as displaying each collaborator's mouse location in real-time and syncing freehand drawings, text, and sticky notes. Additionally, it offers notebook-specific features like synchronized scroll positions, ensuring that all collaborators are literally on the same page. The service is available in both free and premium tiers.

A unified API for all of the open-source tabular AutoML frameworks

Tabular AutoML is where you provide a table (via CSV or whatever) and a column that you want to predict, then a search process finds the pipeline to predict that column most accurately within some time limit. Given the proliferation of open-source tabular AutoML frameworks and the fact that I spent a lot of time working on the closed-source Darwin one at SparkCognition, I thought it would be helpful to have a single API where you can access all of them.

Pose tracking of paintball players

I played paintball competitively for over a decade and was curious if an off the shelf pose tracking algorithm could be used for next gen paintball stats.

Add papers from a BibTeX file to your Arxiv Sanity account

A way to load BibTeX paper citations into your Arxiv Sanity library so that you can find even more papers to read. A lot of times I'll cite a bunch of things in an Overleaf paper and then use this to add the references back to Arxiv Sanity so that I can get recommendations based on it.

Comparison of Google Machine Learning Cloud GPU (in beta) vs local Nvidia Titan X

I had a free trial on the Google Machine Learning Cloud in beta and was curious how their GPUs compared to a Nvidia Titan X in terms of training time on an MNIST benchmark. The summary is that they're fairly similar, but the lag time between submitting a job to the ML cloud and getting output from your code wasn't worth it for me to not just use the local Titan machine.

TensorBoard as a Zipline dashboard

Zipline is a Python library for backtesting and trading quantitative strategies. TensorBoard is a visualization tool provided with the deep learning library TensorFlow. These two can be used together to create a quick and easy dashboard that monitors and compares Zipline backtests.