Lamdu

Lamdu - towards a new programming experience

Project Lamdu

This project aims to create a next-generation live programming environment that radically improves the programming experience.

"Live programming" means that the code is being executed as you type it.

The main idea behind the Lamdu project is that the canonical representation of programs should not be text, but rich data structures: Abstract syntax trees.

Our programming tools, UIs and ecosystems should take advantage and expose this structure. This has some far-reaching implications, which will be described below.

Programming Language

Lamdu's programming language is a functional programming language inspired by Haskell. Language's design decisions are affected by the different trade-offs of live programming and AST editing, and for this reason Lamdu isn't made for an existing textual language.

We believe that Haskell is a good inspiration for Lamdu's language due to 2 main traits:

Rich and Powerful Type System
Haskell has one of the richest and most powerful type systems, which allows a smart IDE to guide the programmer quickly through the space of type-correct programs. In practice, this reduces much of the cognitive burden of writing programs and the number of input gestures/keystrokes required to write or edit programs.

While static typing is considered a hurdle by many programmers who choose to use dynamically typed languages such as Python, we believe that a type-aware structural editor and type inference makes static typing easy to use, helpful and fun.
Effect Typing
Haskell has explicit type-level separation of effectful components from pure components. Real Haskell programs demonstrate that the majority of code can be written as pure components. This allows a live environment to actually execute code as it is being edited, safely, and bring the benefits of spreadsheets to general purpose programming.

How Lamdu's language differs from Haskell:

Explicit argument names

In function applications, arguments are tagged by names rather than using their position to differentiate them (similar to Objective-C).

Most programming languages, which use "positional arguments", were designed for the programmers typing the code letter by letter, and so they refrained from tagging the function arguments. We believe that this tagging helps readability and also provides helpful context when writing the code.
Eager evaluation / explicit lazy-evaluation

The example above uses the logical-or operator (aka "||"), which does not compute the second predicate if the first one is true. In most languages the logical-or operator is built into the language itself (and not its library) as one cannot define it by themselves to not evaluate its right argument when not needed.

Haskell allows users to define control structures such as logical-or and many others, via its behavior of lazy evaluation - expressions are not evaluated until their value is required. But lazy-evaluation-everywhere also has drawbacks (for example it makes memory use hard to predict).

To use eager evaluation but still get the benefits of lazy evaluation (user/library defined control structures and many more) we made the syntax for "deferred computations" (aka functions with no arguments) very lightweight - simply a preceding ◗ symbol. As the user uses the logical-or operator this gets written for them automatically on the operator's right side.
Structural type system
In many programming langauges defining structs/records takes more effort than using "a tuple".

Lamdu, however, makes creating ad-hoc record types easy without requiring the ceremonious type declarations. So records are used instead of anonymous tuples, making it easier to understand the meaning of values.

Lamdu's type system "gets out of your way" with full row-polymorphism and column-polymorphism. Nominal typing (type declarations) is also available and is necessary for types which are hard for type-inference to infer, such as recursive types and types with "higher-rank-polymorphism".

Lamdu Benefits

No Syntax Errors

Editing programs as rich structures means there are no syntax errors

Incremental Type Errors

Type errors are incremental, meaning that type information available from previous, valid states of the program is available to provide better, more localized type errors.

Incremental type checking is much cheaper than full recompilation as featured by mainstream IDEs, allowing a snappy experience.

Type error feedback is immediate, while the edit is still fresh in the programmer's mind.

Type errors are confined and localized to the sub-expression involved, made simple and comprehensible.

Type information is not lost when type errors exist, allowing the IDE to guide the programmer in resolving type errors intelligently.

Better Completions

When types are rich enough, much of the program structure can be inferred from the types.

This affords significantly more powerful and intelligent completions than found in mainstream IDEs.

Live Execution

Frees the programmer from having to trace code execution in their head.

Debugging pure code means expanding sub-expressions that have the wrong result, and following through the sub-expressions that contain the mistakes. This should be a much easier experience than stepping through code.

Safe Refactoring

The IDE has much stronger guarantees about the meaning and structure of the code. This allows refactoring to be completely safe.

Safe refactoring allows programmers to keep the code tidy, without fear of regressions.

Better Collaboration

One of the most difficult things about collaborative development is handling merge conflicts.

The vast majority of merge conflicts are results of non-functional changes: Renames, reformatting, textual movement of code lines between files, etc.

In Lamdu, "names", the "position" of the code and other non-functional aspects of code are separate from the code itself, and avoid conflicts.

Rename conflicts

To get a conflict due to "rename" operations, two developers must rename the same variable to two different names. Even then, the code is still executable, but its on-screen rendering will display a localized conflict.

Formatting conflicts

Formatting is automatic, so there is no way to generate a conflict.

Code movement conflicts

The "position" of code is meta-data attached to the code, helping to find that code and position its rendering.

Since code is not structured into text files, code "position" conflicts are similarly less harmful, less likely and localized.

Change tracking

Instead of heuristically guessing what changed in the code, as traditional version control systems do, Lamdu uses an integrated revision control system and records the changes by the programmer as revisions.

This acts as a record of the developer's intent, allowing the RCS to distinguish, for example, between function deletion and writing of a similar function, and the modification of that same function. The recording of intent helps prevent and resolve conflicts.

Regression Debugging

Integrated revision control and live test cases will allow "Regression Debugging".

When a change causes a regression, the root of the problem can be found quickly, by finding the deepest function application whose result value diverged from the correct version of the code.

Automatic Formatting and Sugaring

Lamdu attempts to take away as much inconsequential freedom from the developer, to free his mind and his key strokes to deal with the parts that matter in the code. Thus, Lamdu does not provide means to edit formatting on a case-by-case basis. Generalized changes to layout rules can be provided, instead.

Additionally, to avoid further stylistic dilemmas, Lamdu uses automatic sugaring of code, as the dual of typical "de-sugaring" done by textual languages.

The code is edited and displayed in its sugared form. The edits to this form are translated to lower-level, simpler edits of the stored language, which is de-sugared. Lamdu uses "co-macros" that capture patterns in the lower-level code and automatically sugar it to canonical form. This frees the programmer from worrying about whether to use sugar for each particular case.

Visualization

Textual languages can have extensible syntax for rendering and parsing values of custom-defined data-types. A rich structural IDE can take this further, and allow defining extensible UI components to render and edit values of custom-defined data-types.

In effect, this allows for more powerful DSL extensions to the language. These DSLs are not just custom data-types and combinators, but also rich UIs with optimized key bindings to work with that DSL.

For example "do notation" can be implemented as a co-macro (see above) to capture the >>= to lambda syntactic form. Then, a customized UI that edits the "do notation" sugared form can be provided, extending the language.

Similarly, a binary tree implementation may be edited with a rich UI that visualizes the trees. The inputs, outputs and subexpressions with tree-typed values will use such rich visualizations, instead of flat textual renderings.

Transparent Build Process

Today's tools treat the compiler as an opaque, black box, which may or may not apply a set of optimizations to the source code, with great or catastrophic results. Some experts are aware of tooling options to display the intermediate steps in the compiler in an arcane, difficult-to-understand notation.

We intend to make the build process, including compilation, be an explicit pure transformation of the code into intermediate and final forms. The steps of this transformation can be visualized by the live programming environment similarly to the visualization of execution of all pure code.

This will allow far easier inspection of the optimization process, taking away much of the "magic" involved in getting optimizations to fire properly.

Similar Efforts

Inventing on Principle

Bret Victor presented amazing demos in his inspiring talk in January 2012. He made quite a convincing case for "live programming" which inspired several projects in this field. Since then he made more impressive demos.

Alive

Alive is a plugin for Visual Studio that provides live coding features for C#. It is a commercial product currently in Beta.

Unison

Unison by Paul Chiusano is another next-generation programming platform project.

Light Table and EVE

Inspired by Bret Victor, Light Table by Chris Granger is an IDE with some "live coding" features. Chris launched the project at Kickstarter at April 2012 and raised more than 300K$ for the project. Soon after Chris founded a company, Kodowa, with Robert Attorri, which joined Y-Combinator's "Summer 2012 batch".

Chris moved on and is now working on project "EVE" where they design a new programming language aiming to bring the power of computation to everyone

Projucer

Projucer by Julian Storer (Jules), also inspired by Bret Victor, is a live programming IDE for C++ using the Juce cross-platform framework.

It integrates with LLVM's just-in-time compiler to provide fast compilation and advanced features similar to those in Bret Victor's demos. It aims to be a "Light Table for 'Real Programmers'".

It initially aims to provide live coding specifically to GUIs, but later for other uses, specifically uses in the Audio niches in which Jules and Juce focus on.

Projucer is not open source and is planned to be a commercial IDE.

The project launched at September 2012 and it looks like Jules was mostly busy with other projects since then.

Subtext

Subtext by Jonathan Edwards is a similar effort from 2004.

In his Subtext 1 demo and talk, Edwards presents a live programming environment of a simple dynamic language.

In his Subtext 2 demo, Edwards presents an inspiring way to edit canonical representations of conditional and type-dispatch code.

The Subtext project was the main inspiration for the Lamdu project.

Conception

Conception by Dmitri Shuralyov is a interactive IDE for Go inspired by Bret Victor, Light Table and Subtext.