Writing Efficient C++ Code (2013) (asawicki.info)

I too am in the "premature optimization bad" camp.

Beyond the low-hanging fruit like ensuring you aren't creating O(n^2) complexity by accident, I think C++ is fast enough/has mature-enough compilers that by the time you're worrying about cache hits materially affecting performance, you're probably also sufficiently staffed and capitalized to pay people to A/B test that performance.

Or even, stuff it into several parallel vectors (structure of arrays instead of array of structures).

While this advice isn't wrong, it is misleading. In my benchmarks std::map beats vector after 9 elements. Less than that and linear search is better but branch prediction and cache loading is very good.

Run your own benchmarks on your own data of course. Also map is not considered the best key value store.

I rarely use C++ but when I do it is for speed. It’s not uncommon that carefully crafted intrinsics can 10x the straightforward naive implementation.

It is probably very domain specific. In robotics for example everything is a zero sum game: CPU, memory bandwidth, GPU, battery life etc ... So it is really a topic, probably true for anything embedded actually. Some other offline applications: HFT, Telco etc.. I wish the GUI apps devs respect more the laptop resources they are running on, don't get me started on the 4 instances of chrome I need to run just for discord, signal etc ...

True, but moving from a list of unique polymorphic pointers to a std::variant gains you at least a 2-3x speed up in terms of TLB and cacheline locality. From there, swapping to SOA will net you another 4-8x, so you're looking at nearly 25x improvement by going data first. That may not matter in the unique case of say, games, where rendering a million entities will dwarf the cost of SIMD processing a million entities, but in something like numerical simulations (fluids) or quant it will be warmly welcomed

When writing code for end-user applications, I think it's mostly true. When it's writing code for a database engine, a game engine, a 3d renderer, or anything else that involves heavy data processing, optimization is the core "thing" often and it might not even be a good enough solution without it. Although, a lot of time even then C++ is good enough even then when picking reasonable data structures to represent the data.

I started writing a [CPU-only] 3-D rendering library in C++ recently, after having written the equivalent in C as a proof-of-concept and an experiment. The reason I decided to write it in C++ after C, is not only because I wanted to tap into meta-programming which is facilitated much better with C++, or that I wanted niceties like procedure overloading, but because some things with C or C++ aren't automagically optimised -- like if you want to leverage struct-of-array (SoA) memory layouts because it allows fewer SIMD (AVX in my case) instructions in the rendering pipeline. You do _not_ get that "for free" just writing a single procedure in C++, much less with C. Both languages are layout-sensitive, I mean this is in part what gives you the speed -- optimising with memory layout for cache locality etc. But you have to do it yourself. Meaning that if you need array-of-struct (AoS) or in fact don't know which path the CPU would prefer, there's no other way than roll up your sleeves and one way or another implement both.

The kicker is, in my case I chose C++ because templates allow me to reuse most of the code in the rendering pipeline _regardless_ of whether I go for AoS or SoA layout. I leverage operator overloading to do vector by matrix multiplication which is implemented in both variants. I do have to specify the desired variant during building, but I've profiled and for Intel x86 AVX in my case SoA is something like twice as efficient because I process ("shade") 8 vertices with 4-5 instructions instead of 1 vertex at a time (still shaded with vectorisation -- just "rotated", i.e in the pipeline axis and not vertex buffer axis).

TL;DR; C++ gives you plenty fast by default, but it's not always enough. The difference between 5 and 15 frames per second, well, makes all the difference -- our eyes are only fooled once the frames-per-second rate goes sufficiently up, anything below an acceptable threshold and it's completely different experience. You then either sacrifice resolution or level of detail etc, or decide to squeeze more from the language by helping the compiler.

Then you don't work on a product that has any scale <shrug>.

Definitely need to optimize a bit for games and huge scale web apps. We've been finding big optimizations in our app recently. App works without them because we can scale horizontally but cutting CPU usage by 30% by eliminating redundant work and reducing copies of big objects? Why wouldn't we want to do that? This isn't even fancy algorithm stuff, mostly just shoddy initial implementations by 100s of eng working on a codebase over 7 years (not even that old). Stuff like that creeps in.

This is a sign you might be more productive in a higher-level language.

This follows Rob Pike’s 5 Rules for Programming

https://web.archive.org/web/20250201145327/https://users.ece...

There’s code where there exists a concept of “fast enough,” and code where there is no such thing.

Then why are you using C++? Java/C#/Go are already fast enough for general application development. Why would you accept the footguns if not for performance?

Depends on your field. When one microsecond is considered "hellishly slow", you might reconsider

"blazingly fast" as in how much trading rules could you apply to 10Gbit/s stream of stock market data on one core? On one socket?

How 100Gbit NICs could your filter through your stateful firewall at line speed? And with 64 byte packets?

It really depends a lot on the domain.

In (soft) realtime audio programming, your audio callback might only have a time budget of 1.3 milliseconds. Everytime you exceed that limit, you'll hear a dropout. That's when you'll start to optimize the hell out of your program :)

Learn which instructions SIMD nicely (sqrt / fabs, etc). Use ternaries in loops for masking. Use trig identities and lookup tables (don't recompute sin(3t) when you can use two vector multiples using a table of sin(t) eg. sin(t) * sin(t) * sin(t)). Use divisible constexpr constants in loops to eliminate the SIMD tail. Be careful with type casts and floats. `float x; x += 0.5` will introduce *cvt instructions even if the compiler statically knew better otherwise (use 0.5f). Compile with --fast-math and friends so errno doesn't invalidate your SIMD pipeline.

    #define fn __attribute__((used))

    fn void copy1(int* to, const int* from, const int size)
    {
        for(int i = 0; i < size; i++)
            to[i] = from[i];
    }

    fn void copy2(int* to, const int* from)
    {   
        constexpr int size = 1024;
        for(int i = 0; i < size; i++) 
            to[i] = from[i];
    }

    fn void copy3(int* restrict to, const int* restrict from)
    {
        constexpr int size = 1024;
        for(int i = 0; i < size; i++) 
            to[i] = from[i];
    }

    gcc test.c -c -O3 && objdump -d ./test.o

They're a bit old and missing some details, but I like Agner Fog's manuals.

https://www.agner.org/optimize/

Do you have any recommended essential reading for this?

How? you can write exactly the same low level code today.

I think it has become EASIER: for instance, since C++23 Rust-like move semantics can be used, which provides the compiler with extra information that can be leveraged for the generation of better code.

Or take constexpr - it permits to move computations to compile time that are complex and in older versions either had to be done at runtime, or an ugly workaround had to be used (e.g. assigning a mysterious literal pre-computed in another run or by hand).

Writing clear, concise, and efficient code in C++ has never been simpler or easier. The improvements in C++ over the last 15 years have been qualitative.

So many complex, esoteric, and difficult to maintain incantations that used to be required for efficient code generation are no longer necessary.

You heard correctly. This discussion is about C++ though.

Yes, but the numerical backend of the python libraries are written in lower-level languages. For example, pytorch uses a C++ backend.

Python is often used as (very useful!) glue for calling CUDA, C, C++, Fortran, etc… codes.

So, if you are thinking about the sort of “business logic” that’s often Python, but the performance comes from the parts that are usually not.