I'd be inclined to agree, but my 16gbps i5 520m compiles Rust just fine. Small and medium-sized desktop applications aren't constrained by bandwidth as much as they are CPU-bound.
I could easily imagine AI model training blowing up a memory bus, but every machine that isn't an M1 Pro or M1 Max is going to delegate that to a dGPU that is bound by PCIe bandwidth, not memory. Even then, properly-optimized CUDA programs will calculate most of the work on your GPU itself, constraining you to the 300-something gbps that your GPU is capable of.
Regardless, neither of these workloads are something that I'd imagine the average Macbook user doing on a regular basis. The only "real world" workload I can imagine that would exercise it is insanely high-res video encoding, which again, is only really going to be taken advantage of by a handful of users.
Discussing needs of an average Macbook user is a moot point as they would not purchase a maxed out M1 Max. I was referring to specific use cases of a specific target group of users with specific examples who can benefit from specific hardware architecture solutions and about being able to put them into a practical use today.
> I'd be inclined to agree, but my 16gbps i5 520m compiles Rust just fine. Small and medium-sized desktop applications aren't constrained by bandwidth as much as they are CPU-bound.
Gbps refers to the memory throughput, not to the memory footprint. A Haswell CPU equipped laptop with 16 Gb of RAM can compile Rust applications just fine; the difference is the time. Compile times are the single important delineating factor for software engineers.
> Even then, properly-optimized CUDA programs will calculate most of the work on your GPU itself, constraining you to the 300-something gbps that your GPU is capable of.
With all due respect, it is exceptionally unusual to train ML models directly in CUDA, which is a low-level proprietory Nvidia API. Of which CUDA is one, and there are several others as well (ROCm, OpenCL, CoreML, Metal etc), and they are swapped out for one another depending on what the underlying hardware actually is. ML training is done via the use of high-level ML frameworks (Tensorflow, NumPy, SpaCy, Pandas, SPARK and others), but not in CUDA – it simply almost does not exist.
Lastly, whilst the ML training undertaken on a high performant and an exorbitantly expensive dGPU itself would be substantially faster compared to 32 M1 Max GPU cores, one would yet be constrained with the PCIe bus data transfers speeds: ~32Gbps for PCIe v4.0 in the x16 lane setup (the high end of the commodity PC hardware today), and ~64 Gpbs for PCIe v5.0 in the x16 lane setup (not even unavailable for purchase today in PC laptops and desktops). With both being a far cry from the 200 Gbps of a single M1 Max CPU cluster.
I could easily imagine AI model training blowing up a memory bus, but every machine that isn't an M1 Pro or M1 Max is going to delegate that to a dGPU that is bound by PCIe bandwidth, not memory. Even then, properly-optimized CUDA programs will calculate most of the work on your GPU itself, constraining you to the 300-something gbps that your GPU is capable of.
Regardless, neither of these workloads are something that I'd imagine the average Macbook user doing on a regular basis. The only "real world" workload I can imagine that would exercise it is insanely high-res video encoding, which again, is only really going to be taken advantage of by a handful of users.