Python's experimental JIT to be removed (lwn.net)

FWIW, I definitely can find a bunch of both PEPs and JEPs for internal optimizations, random examples:

https://peps.python.org/pep-0509/

https://openjdk.org/jeps/147

They do:

https://openjdk.org/jeps/197

https://openjdk.org/jeps/515

Not every optimization requires a JEP, but certainly something as big as "add or remove a JIT" would 100% require a JEP.

Folks, numba is great. To first approximation: write your python function to "look like C"[1] and work over the numpy arrays you probably already have and understand, put a "@jit" decorator on it, and you're done. It JITs and runs your code natively at native performance.

With some extra effort, there are features for SIMD vectorization and thread parallelism, and even GPU backends (that, full disclosure, I've been pretty frustrated by -- but they do work).

[1] Which is to say, don't touch Python interpreter data structures. No dicts or objects or lists or strings, etc... If you're bound by the interpreter[2] you're bound by the interpreter and nothing is going to help you.

[2] A corrolary here is that the kind of C code numba is NOT good for is "following pointers between structs", which is something that can't be expressed in either python or numpy.

Maybe also then, importing module as Cython?

  import django with Cython

Python already compiles to bytecode (the .pyc files are a cache thereof). It's just that interpreting even the simple bytecode instructions can take dozens of clock cycles each. I imagine that a lot of this is not in the interpreter loop (which has seen quite a bit of optimization, IIRC up to the level of "predictive" gotos within a big switch statement within a loop) but just unwrapping objects and moving values to and from registers, dealing with the fact that there is basically never any cache coherency for anything etc.

Basically I'm not convinced that writing out the set of machine-code instructions needed to interpret the bytecodes would be a big win, because it takes too many instructions for simple tasks.

Heh, how about how `__class__` works? Not that I recommend actually doing this, but you can literally change the type of an already instantiated object by reassigning `__class__`:

    >>> class Dog:
    ...     def hi(self):
    ...             print("woof")
    ...
    >>> class Cat:
    ...     def hi(self):
    ...             print("meow")
    ...
    >>> dog = Dog()
    >>> dog.hi()
    woof
    >>> dog.__class__ = Cat
    >>> dog.hi()
    meow
    >>> isinstance(dog, Dog)
    False
    >>> isinstance(dog, Cat)
    True
    >>>

Have you seen Ruby?

JavaScript also has monkey-patching, and it's also very common. E.g. SolidJS replaces objects with dependency-tracking proxies.

There are really no fundamental language reasons why Python should be more complex. From what I heard from a JIT developer, the main issue is not the code itself but the module system and code loading.

JavaScript was designed from the start to operate inside a sealed environment with only few external handles, with the code loaded upfront or added on-demand via "eval". Not so for Python, where native code modules can (and will) result in code loaded dynamically and it's hard to track what depends on what.

All just as dynamic as Python, and with quite usable JIT compilers.

Or for that matter, PyPy and GraalPy do exist.

someone please think of the monkey patchers!

If having to put all that in writing and get a bunch of people to agree feels too burdensome to you, maybe that’s a sign that you would be happier working on something other than core changes to the world’s second most popular programming language.

(edit: That wasn’t meant to be a jab at the core developers working on the JIT. Based on what’s in the public discussion threads, they have been handling things very well throughout the process. It’s unfortunate that the SC let the project get this far before realizing the project had grown well beyond its original PIP’s mandate, and I’m sure that is deeply frustrating to the people leading the project. I know I would not be handling the setback nearly as gracefully as they are.)

HN when an organization achieves its intended goals: this can't be bureaucracy because it's good.

Pypy does provide faster pure Python. Its a compatible implementation with JIT.

PyPy wasn't great for me because its startup time interfered with the main usecase I wanted it for in one instance and in another there bugs that made it not quite compatible enough. Money could solve the bug issue.

On the other hand I did get pygame working with it at one point and that seemed like a nice scenario.

https://pyronaut.io/performance/

> Latency lines come from the load ramp (+25% per 15-second step); throughput from 45-second validation steps (+2% each) after a 180-second warm-up. Each framework ran twice and we show its best run. No profilers were attached.

So that's a "realistic" web serving benchmark if you're willing to grant the use of a GraalPy-optimized framework rather than Flask/FastAPI.

Warmup times for other benchmarks are currently quite unstable because there's a big focus on interpreter performance at the moment, which is one of the main influences on warmup. There's a blog post here from August about how the compiler is being taught to recognize and optimize bytecode interpreter loops, to get results that normally require handcrafted assembly:

https://medium.com/graalvm/escaping-the-giant-switch-dec20b5...

So as you know already, warmup was historically a weakness of PE based stuff - although in this case the fair comparison is with CPython which never warms up at all - but a fast interpreter will make a big difference. In some benchmarks the GraalPy interpreter is already faster than the CPython interpreter, which is heavily optimized.

I think if the Python community wants an implementation that has a proven JIT and where the remaining work is easily scaled compatibility testing, then there's one available and ready. Whether they're calling into C or Rust or FORTRAN or Java shouldn't make a big difference in principle.

Now, I've met lots of people who think that binding to Rust will always be the fastest thing to do but that underestimates Java significantly, especially once you factor in the fact that GraalPy can optimize out interop overheads.

Consider LLM tokenization. It's one of the most heavily optimized pieces of code on the planet. Yet, toknroll written in Java can beat tiktoken which is written in Rust:

https://github.com/qxoticai/qxotic/tree/main/toknroll

Moreover the Rust code has some unsafe{} blocks as Rust often does, whereas toknroll has no unsafe code anywhere.

So the scripting language communities need to think hard about where they go next, IMHO. DHH already got up and announced at a Rails conference that Ruby is over, you should use Rust for everything. I don't think that's quite true, but it's going to be a meme that with agents you should just pick the fastest language and go. If Python/Ruby/etc want to stay relevant then they need to care a lot more about their performance story. Just being more pleasant to read and write for humans doesn't cut it anymore. CPython is clearly not going to happen in this regard, so alternative implementations need to start being taken a lot more seriously than they were in the past.

They are Python libraries. Don't make the mistake of screwing up the cause and effect here. C, Rust etc - those are the replaceable parts of the chain. There is an official C++ pytorch library and virtually nobody uses it for ML Research. People want to code in python and build those libraries so they can.

Today there are far more people using it, and everyone has a different perspective on what it is. No longer is it a niche glue language, now it's a backbone of AI technology at major tech firms, so it's understandable they'd like it to be faster on its own.

Probably also during the 2 to 3 migration, but that goal appears to have now been abandoned.

I wonder how the authors of other implementations like pypy will be able to keep up as the reference implementation becomes more and more complex.

https://peps.python.org/pep-3000/ - PEP 3000 says it will be a derived from the Python 2 codebase and not a complete rewrite (referencing Joel Spolsky's views on complete rewrites and Netscape Navigator).

https://peps.python.org/pep-3099/ - PEP 3099 is changes Guido van Rossum rejected from Python 3 (including := as a binding operator), parallel iteration, programmable syntax, parser going beyond LL(1), changing the for/else construct.

https://peps.python.org/pep-3100/ - PEP 3100 is things he set as goals for it, things like all strings are Unicode and byte arrays are a separate type, optional type annotations, 'exec' and 'print' change from statements to functions, None/True/False become keywords, old-style classes are gone, changing inequality comparison from <> to != , require C99, drop BeOS support, drop old standard library modules, and more.

actually, more changes than I remember.

More digging led to this: https://peps.python.org/pep-0659/ which seems to be what the effort is organized around?

Python's object semantics are broadly similar to Javascript, bar the prototypical inheritance chain, but with the extra complexity of potentially needing to handle dynamic `__*attr__` overrides. Optimizing for this would require a tree of hidden map objects much like JS JIT engines use. I dug around the codebase a while back looking for something like that but didn't find it. `OrderedDict` seemed promising but after more investigation that turned out to just be adding property ordering to dictionaries.

The thing is, you can build that entire hidden type structure out, AND optimize against it, without ever having to mess with things like bytecode replacement. Once you have the hidden type model, you can modify the property lookup algorithm in the VM itself (as in cpython c code) to use some annotation structure hanging off of the bytecode to look up inline caches of type-guarded fastpaths. It would be an optimized fastpath that integrated directly into the C code that interprets the GETATTR opcode.

His current focus seems to be speeding up GC and free threading.

>> For example, rather than proposing one single concrete JIT implementation, it may make more sense for the PEP to describe a JIT infrastructure that can support multiple implementation strategies.

https://facebookincubator.github.io/cinderx/#performance

V8 predates strict mode, and handles all the exec/eval, monkey patching and object system abuse you can do in JavaScript just fine. Common Lisp is one of the most dynamic languages in the world and allows you to do ridiculous things in terms of meta-programming and essentially building domain-specific languages into your program, yet it too manages to have ahead-of-time compiled implementations that are very performant. Also PyPy exists.

There are definitely some technical aspects that make it hard to make Python fast, but they are not insurmountable. The reason CPython is slow today is simply because performance has not historically been prioritized by its developers.