Visual Studio has long been recognized for the quality of its IntelliSense (code analysis and suggestions) across all languages, and has had support for Python since 2011. We are pleased to announce that we are going to be making the Python support available to other tools as the Microsoft Python Language Server. It is available first today in the July release of the Python Extension for Visual Studio Code, and we will later release it as a standalone component that you can use with any tool that works with the Language Server Protocol.

Background on IntelliSense and Language Servers

Ever since the days of Visual Basic, one of the core features of the Visual Studio series of IDEs has been IntelliSense: auto-completions for variables, functions, and other symbols that appear as you type in your code. Through a clever combination of static code analysis, precompiled databases and UI overlays, developers are regularly blown away at how productive they can be with an editor that truly understands their code.

Fast forward to today, and IntelliSense is still one of the most important features out there. More tools are requiring users to write code, and completions are practically a necessity in these editors. However, writing the static analysis necessary to provide a great experience is difficult, and most implementations are very closely tied to the editor they work with. Enter the language server protocol.

Language servers are standalone programs implementing the language server protocol, and were created to work with Visual Studio Code. Editors can start running a language server and use this JSON-based communication channel to provide and request information about the user’s code. All of the analysis and "smart" operations are handled by the server, allowing the editor to focus on presentation and interaction with the user.

Visual Studio Code uses language servers for most of its supported languages, including C++, C# and Go. From the editor’s point of view there are no differences between these languages - all the intelligence exists in the language server. This means that it is easy to add support for new languages to Visual Studio Code, and it does not require changing the editor at all. Language servers can also be used with plugins for Sublime Text, vim and more.

Introducing the Python Language Server

Previously, Python IntelliSense in Visual Studio was very specific to that IDE. We have been developing this support for nearly a decade. It has an impressively deep understanding of the Python language, but only Visual Studio users have been able to enjoy this work. Recently we have been refactoring our implementation to separate it from Visual Studio and make it available as a standalone program using the language server protocol.

From the point of view of the editor, language servers are a black box that is given text and gives back lists of more text. But the black box normally contains a process known as static type inferencing, where the language server determines ("infers") the type of each variable without actually running the code. For statically-typed languages, such as C#, this is often as simple as finding the variable definition and the type specified there. However, Python variables can change type any time they are assigned, and assignments can happen almost anywhere in any of the code that is run. This actually makes perfect static type inferencing impossible!

(Technical aside: Variables are often thought of as "holes" into which only compatible values can "fit", where the shape of the hole is determined by its type. In Python, variables are names that are attached ("bound") to the value when it is assigned. Assigning a new name always re-binds the value regardless of whether the type is the same as the previous one. So just because you see "self.value = ‘a string’" in one place doesn’t mean that "self.value" will always be a string.)

Our Python Language Server uses iterative full-program analysis to track the types of all the variables in your project while simulating execution of all the code in your project. Normally this kind of approach can take hours for complex programs and require unlimited amounts of RAM, but we have used many tricks to make it complete quickly enough for IntelliSense use. We have also made the tradeoffs necessary to provide useful information despite it not being possible to perfectly infer all types in a Python program.

The end result is that we have a black box that takes Python code and provides all the information your editor needs for tooltips, completions, finding definitions and references, global variable renaming, and more. For performance, it runs with .NET Core on Windows, macOS and Linux, works with Python 2.5 through to Python 3.7 and supports the latest language features such as async/await, type annotations and type stub packages (including typeshed, a copy of which is included with the language server). It performs incremental updates as you type, and is already proven as a core feature of Visual Studio.

Benefits for Python in VS Code

Our July release of the Python extension for Visual Studio Code will include an early version of the Python Language Server. Features that are new for VS Code developers in this release include:

• Syntax errors as you type in code
• Warnings when modules are not found
• Using typeshed files to fill in missing completions for modules
• Improved performance for analyzing your workspace and presenting completions
• Ability to detect syntax errors on your entire workspace, rather than just the current file.
• Faster startup times
• Faster imports
• Better handling for a number of language constructs

All of these are already available in Visual Studio 2017 or will be in the next minor update.

Having a standalone, cross-platform language server means that we can continue to innovate and improve on our IntelliSense experience for Python developers in both Visual Studio and Visual Studio Code at the same time.

Be sure to check out our VS Code release announcement for more information. The standalone release of the Python Language Server will follow in the next few months, and will be available under the Apache 2.0 license.