This Week in Mojo 2023-05-19
Mojo Playground Release
- Added missing dunder methods to PythonObject, enabling the use of common arithmetic and logical operators on imported Python values.
- PythonObject is now printable from Mojo, instead of requiring you to import Python’s print function.
- Issue #98: Incorrect error with lifetime tracking in loop.
- Issue #49: Type inference issue (?) in ‘ternary assignment’ operation (FloatLiteral vs. ‘SIMD[float32, 1]’).
- Issue #48: and/or don’t work with memory-only types.
- Issue #11: setitem Support for PythonObject.
- Maxim Zaks did a blog post on counting chars with SIMD in Mojo
- Abhinav Upadhyay did a substack post titled Mojo: The Future of AI Programming 🔥
- Code to the Moon did a live demo and introduction to Mojo titled All-in-one C++, Rust, AND Python Successor?
Mojo Team Answers
The standard Pointer is very unsafe and can lead catastrophic scenarios
I agree, the Mojo Pointer type is currently "too sharp and pointy" 😀. In my opinion, we should rename it to
UnsafePointer and make some other changes to make it not have to be something that people reach for immediately, just like in C++ or Rust you should use higher level collections, and not jump right to unsafe features.
We have to implement array bound checking for our array/slice types, we just haven't solidified them due to missing features (notably traits)
On community, this dovetails with our open source plan. We're getting a bit crushed under lots of different kinds of interest right now, but I'd love to open up more code, enable pull requests etc, that's mostly blocked on logistical work and that we're being crushed in various ways. We have a Mojo developer advocate role open that will help us sort that out.
currently an unimplemented feature We don't have a final name here, Guido recommended that
Protocols as term of art in python already, but we'll need to loop back around and make a decision when we get there.
On the implementation, we'll need some work to build out
help(Int) (where Int is a struct, not a class). I don't see us prioritizing that in the next month or so, but it is super important for us to do that over time. We have ways to do that without adding a field to Int 🙂 etc, so that should be fine. It depends on Traits/Protocols which is on our roadmap
CPython Implementation Details
You're quite right about CPython. Mojo takes a different implementation approach: ignoring C extensions for a moment, the core compilation model for mojo is to compile to native code, and use ownership optimizations, and more modern data layout approaches to avoid heap boxing all the things, and therefore reference counting them. In CPython for example, a lot of reference counting traffic is required for simple integers and short strings etc.
Mojo solves this in several ways:
- compilers: you get a lot of performance by not going through an interpreter, using register allocation etc.
- unboxing things: our default "object" is still naive in many ways, but has inline storage + variant for small types like integers to avoid indirections, refcount overhead, etc.
- types like Int are put in cpu registers etc, which give a massive performance uplift vs that.
Now you can't ignore CPython and can't ignore c extensions. Good news, MLIR and compilers can do more than one thing 😃, and so we can talk to other ABIs and handle other layout constraints. We haven't built a proper "talk to c python extensions" directly from Mojo subsystem, but when we do, it will have a GIL because c extensions require it, just as you say.
Similarly, when you import a cpython module, you get the cpython interpreter in the loop, which has a gil (and its datalayout etc) implicitly.
The cool thing about mojo is that you don't pay this overhead in pure Mojo code, so if you care about performance you can incrementally move Python code -> Mojo and you can adopt new features for performance ... but only if you care about performance! If you don't, hack on and do so without caring, and all is well.
yes, definitely, we want Mojo to go everywhere, and deploying to small devices is part of our design. One step at a time though 😀
It will be nice to change the current rebind parameters from [dest, src] to [src, dest] since its more intuitive that the other way around. The current signature is rebinddest_type, src_type
The current way works better with parameter inference, because you can call it with
rebind[dest_type](src_val) and have src_type inferred from the argument.
Pytorch on Different hardware
We outperform PyTorch across a large range of hardware (Intel, AMD, ARM etc) see performance dashboard and swap around the Instance Types
We support quantization and it will support many other HW types like edge deployments
Inference Engine Frameworks
It’s a unified engine that enables multi-framework support - many users aren’t just using PyTorch (TensorFlow, JAX etc)
It integrates natively with Mojo 🔥 for a completely new high performance programming model that enables many things outside of just pure model execution performance
comptime is really obvious to Zig folk, but that's not really our audience. You're right that
alias may not be the right word to use here either. Aligning this around "parameter" could be a good way to go, but I'm curious if there are other suggestions.
Once nice thing about "alias" is that it is more obvious for the trivial cases like alias my_magic = 12312 or alias Int8 = SIMD[DType.int8, 1]. That doesn't make it the right thing, but it is a nice thing.
If we replaced the keyword "alias x = 42" with "parameter x = 42", then we can say "it's a declaration of a parameter" and that "parameters are all compile time expressions."
alias (regardless of what it is called) is a declaration of a thing. We need spoken vocabulary for programmers to describe these things. It isn't just about encoding things in source code for the compiler, it is allowing humans to communicate ideas as well.
Also, "let" values are not aliases. They've very different. A let isn't mutable after it is initialized, which is a flow sensitive property, e.g. this is allowed:
let x : Int if cond: x = foo() else: x = bar() use(x)
which isn't allowed for aliases.
MLIR and LLVM
Reading the documents on MLIR related APIs, I feel that the style of these APIs seems to be quite different with Python
Indeed, the MLIR integration hasn't been polished or designed to be pretty - we've focused primarily on making it fully capable and unblocking our needs. The idea for it is that only MLIR experts would be using this, but then they'd be wrapping user-facing Pythonic types and methods around them (e.g. like OurBool wraps i1). that said, we can definitely improve this in various ways, we just can't do so at the loss of fidelity/expressiveness.
I wonder if it is possible to make Mojo more extensible such that it can also create new didacts?
This is also something we're likely to look into in the far future, but isn't a priority right now. Also, as mojo opens up more, it would be great for community members to poke at this.
MLIR code with unknown dialects
The mojo compiler has a number of internal dialects, including
kgen, but they aren't documented yet. They are very much internal implementation details of the compiler and change all the time. I'd recommend sticking with the llvm and other dialects that are more stable.
i32 vs si32
Python programmers will probably be more familiar with the i32/u32 syntax.
Yeah, for the core language types, our audience are general programmers and Python folks, not MLIR nerds 😉
We want things to be clear and unambiguous, compiler folk can deal with naming mapping. We will discuss.
would it ever makes sense for Mojo to also support signless integers?
I don't see a benefit to that. It would mean that we couldn't use the standard Python operators (which imply sign behavior, e.g. on divides). Signless integers are good for compilers because they want canonical forms, but users want operations that work on types. It's a bit of a different concern.
Optimization via MLIR
Mojo is a gateway to the whole MLIR ecosystem. It is entirely plausible that the matmul implementation for a particular piece of hardware just calls a few MLIR operations.
We can only say that we're working on accelerators and that is core to the mission, but can't talk about that until we're ready to talk about it 😀
Compile Time Optimizations
Mojo's compiler is not going to be magic. If you write matmul as a triply nested for loop, you will get a triply nested for loop on all hardwares (barring LLVM optimizations).
The general idea is that Mojo's compiler is not going to perform some magic to optimize the code you are generating, but the language provides all the facilities to write that magic in a portable way as just Mojo code. Today, that magic is bundled into a handful of higher-order functions, like parallelize and vectorize_unroll, and as time continues, Mojo will ship with more "batteries" that mean most developers won't have to worry about SIMD, unrolling, etc. You just need to slap a few decorators on your functions/loops and call a function.
Python keyword compatibility
For now, we need to get Mojo from 0.1 to at least 0.7 (conceptually, we have no specific versioning planned), at which point we'll understand more of what we're dealing with, and have broader developed relationships with the python community.
Also, my understanding is that Python3 generally doesn't take hard keywords for various compatibility reasons, even things like "case" are a soft keyword. If that is true, then we may be fine.