How GitHub Copilot Works: Context, Memory and Network Traffic

Hello, Rafael here - every week I cover interesting challenges and developments that I’ve come across recently through the lens of an engineer building AI systems.Subscribe to get weekly issues 👇There has been a flurry of AI-powered apps and AI features in the last couple of years. Incumbent players like Slack have swiftly added AI features to its roster. For AI-native ones like Cursor, Notion, ChatGPT Desktop and Claude Desktop, AI was always part of the raison d’être. The more AI features these apps released, the more I became inclined to look at their inner workings. Hopefully I would be able to uncover a bit of what’s running under the hood; at the very least, I would learn one thing or two about desktop app development.Coincidentally, I noticed I started exhausting my Copilot credits earlier and earlier each month. This ended up pulling me towards selecting a main candidate for my experiments. I decided to dive deep into VS Code and Copilot.Common amongst all of the apps above is the fact that they are built using Electron. Electron is a JavaScript framework which helps developers build and distribute desktop applications. In layman’s terms, it works by bundling a Node.js runtime along with HTML, CSS and JavaScript artifacts, which are then rendered via Chromium.List of apps using Electron. I know, it’s a lot. SourceThis removes the need of having multiple codebases in native languages for different platforms (for instance, C# for Windows and Swift for macOS), making it easier for developers to build desktop applications that run across multiple platforms from a single codebase. (Native modules and certain packaging steps still often require per-platform handling, but the bulk of the application logic is shared.)Because they share Electron, they share a rough architecture, which means whatever I learned probing one should transfer to the others.Lighthouse Newsletter is free :) support my work by sharing it with friends colleagues would be interested 👇ShareMy first instinct was to just skim through VS Code source and see if it could answer my questions. The problem was that I didn’t have a full set of questions yet - and hunting for them across millions of lines of code would cost me either too much time or too many tokens. Source code tells you what an app can do; discovering what it actually does at runtime is more challenging. Especially when you still don’t know what you’re looking for.There was a second problem. VS Code is an exception amongst the apps I had started with: its source code (or at least the majority of it) is open. This is not the case for Claude, ChatGPT, Codex, Notion, and Slack. That started pushing me toward the reverse engineering route: passively watch the traffic first, let the requests and responses tell me which questions would be worth asking, and only then go to the source to confirm (or disprove) what I was seeing.It meant getting my hands dirty with Electron’s architecture and network stack - skills that wouldn’t hurt to have afterwards.By now we know that Electron apps ship with Chromium. The browser provides the rendering engine for the application’s web based UI, but it also provides a network stack that renderer processes can use for HTTP and WebSocket connections.This is a common (and recommended) option for enabling apps to speak to a remote backend, but it is not the only one. Applications can also make HTTP requests using Node’s http/https/fetch. Which path the request takes becomes important when you’re trying to intercept it.In some cases, like with VS Code, the application will have a decoupled architecture, where there’s a separate group of processes that acts as an extension host. This helps maintain clear boundaries between distinct responsibilities; in the case of VS Code, a clear boundary between UI, code IDE functionality and plugins/extensions.One of the classic ways to intercept an application’s network traffic is by standing up a proxy server, and configuring this application to use it.The proxy acts as a man-in-the-middle (MITM): it intercepts HTTP requests from a client, forwards them to a server, and relays back the server responses to the client.Fun fact: a similar approach is quite common in corporate network environments for traffic inspection purposes, especially in highly regulated industries. Fittingly, one of the main open source tools used for this is called mitmproxy, which we will use in the next steps.An important detail is that most of the network traffic nowadays happens via secure HTTP (HTTPS). This means traffic is encrypted using TLS. By trusting mitmproxy’s locally generated certificate authority (CA), the client can accept the certificates mitmproxy generates on the fly for each destination. Instead of a single end-to-end encrypted connection, you get two: one between the application and mitmproxy, and another between mitmproxy and the destination server.mitmproxy can therefore decrypt the request, inspect it, establish a separate TLS connection upstream, and forward the response back to the application.If you don’t want to follow along with the code and would just like to see the results, feel free to skip this section.On macOS, the simplest way is to use brew:We need to change some settings in VS Code to route its traffic via mitmproxy. You can change these settings by using the hotkey combination Cmd+Shift+P and searching for User Settings. You will then need to make sure that the settings below have the following values:Http Proxy: http://localhost:8080 (mitmproxy will be listening for connections at this port)Http Proxy Strict SSL: unchecked (we want to skip verification of mitmproxy’s certificate against a list of CAs)Http: Proxy Support: override (force proxy support for extensions)VS Code settings for routing network traffic into mitmproxyAfter making these changes, be sure to restart VS Code.The final step before getting started is starting up mitmproxy’s web UI:Give it a few seconds and you should start seeing some network traffic from VS Code flowing through it.You will notice some text fields on top. You can ignore most of them for now; the most useful is the first one, Search. This field provides powerful search capabilities like keyword search, regex, etc. For instance, if we are particularly interested in the requests made by VS Code to its Extensions Marketplace API, we can simply use marketplace as a filter string.This will match all requests to https://marketplace.visualstudio.com and all its subpaths.mitmweb - Sample list of captured flows from VS CodeIt could be that even after all this dance, your proxy still doesn’t capture extension traffic. This can happen if VS Code’s Extension Host process group becomes stale. To confirm this, run from the terminal:Confirm the date that is displayed. If it’s not the same date and time from when you restarted VS Code, the extension host process is most likely stale. Solving this is simple:In VSCode, open the Command Palette (Cmd+Shift+P)Run “Developer: Restart Extension Host”Re-run your ps grep afterward - you should now see new PIDs with today’s timestamp for Code HelperQuickly skim through the network requests from VS Code in mitmweb and you will notice that the majority of them are related to either GitHub or Github Copilot. Before we hit a single key in VS Code or in the Copilot extension, some HTTP requests are made. Requests made by VS Code and Copilot during the bootstrap stage can be allocated into one of the following categories: Auth & Session, Config & Policy, MCP Registry, Repo & Session Context, Model Discovery and Recent repos.In the next paragraphs, we discuss what I found out about each of these types of requests: what’s included in headers and payloads for requests and responses.A high level view on the percentage of network requests made by VS Code during bootstrap stageThis is the first thing done by Copilot at startup. It fetches an OAuth token, exchanges it for a short-lived token, and validates the user’s entitlements. The flow is quite a regular OAuth one; it is described in the diagram below.Sequence diagram describing Copilot’s OAuth authentication flowBefore making any LLM requests, Copilot checks which models and agent capabilities are available for your account/plan. There are two separate kinds of requests. First, a request is made to /models. This initial request returns a general list of models which are available within Copilot. Then, a second request is made to /agents/swe/models. This is a specific request to find out which models are available for agentic capabilities related to Software Engineering (SWE).Sequence of requests made by Copilot as part of its model discovery stage. Two requests: one for general models available, a second one for models specifically suited for agentic capabilities related to Software Engineering.Post bootstrap is where things get interesting.I selected Auto mode for all the Copilot tests in this experiment. After I sent each message, I was able to capture a request to a /models/session/intent endpoint before any model answered.What’s happening here is: your prompt gets scored against possible intents, such as code-gen, debugging, reasoning and tool-use. The intent classification outcome helps Copilot define which of the available models will fulfill the task.Sequence diagram showing separate requests being made for model and capability discoveryThis is not really a secret; such behaviour is described in Copilot’s documentation. Still, it was fun to see the actual requests and responses behind it.I started to play around with inline completions and ghost text, watching what was being sent via HTTP. I already knew inline completions inject the current file into prompts as context; that’s how it’s supposed to work. So no surprise here thus far.But I still wondered about what else got sent, so I did a small test. I dropped a fake secret into a .env file - the infamous file all of us kids are told not to commit, but