Speeding up surface-state calculations
SurfaceGreensFunction and FermiArcMap solve a Lopez-Sancho
recursion at every k-point on every energy. For realistic problems
that’s tens of thousands of dense complex-matrix factorizations — so
they’re the most CPU-hungry calculators in the package. Two knobs
on each class control how that work is laid out:
``n_jobs`` — k-point parallelism across worker processes.
``chunk_size`` — energy-axis batch size on each k-point.
Both default to safe values, so existing code keeps working
unchanged. Setting n_jobs=-1 is the single biggest win.
TL;DR
Anywhere you call SurfaceGreensFunction or FermiArcMap, just
add n_jobs=-1:
sgf = SurfaceGreensFunction(
model, surface=np.eye(3),
energies=np.linspace(-1, 1, 201),
k_path=[[0, 0.5, 0], [0, 0, 0], [0.333, 0.333, 0]],
k_labels=["M", r"$\Gamma$", "K"],
n_jobs=-1, # <-- use every core
).run()
Results are bit-exact relative to n_jobs=1 — no precision trade-off.
How much faster?
Measured on a Bi2Se3 slab (124 Wannier orbitals, slab dim 248), 16-core CPU:
Run |
Serial |
|
speedup |
|---|---|---|---|
SurfaceGF, |
28.9 s |
9.3 s |
3.1× |
SurfaceGF, |
~480 s |
79 s |
6× |
FermiArcMap, |
14 s |
3.5 s |
4× |
Bigger problems get bigger speedups because the fixed worker-startup cost (~1 s/worker on macOS, faster on Linux) amortizes away. On a 32+ core Linux machine, expect 10-12× on the larger problems.
n_jobs — k-point parallelism
Each k-point of the surface Green’s function is fully independent of
every other, so fanning them out across worker processes via
joblib is essentially free correctness-wise.
|
Behavior |
|---|---|
|
Serial. No worker processes spawned. |
|
Use every physical CPU core on the host. |
|
Use exactly |
Each worker pins itself to one BLAS thread to avoid
oversubscription — without that, N workers each spawning N
BLAS threads would thrash the cache and run slower than serial.
A small caveat on macOS / Windows: joblib spawns workers fresh
(not fork), so each one re-imports torch on startup. That adds ~1 s
per worker, paid once per .run() call. For runs longer than ~10
s the overhead is negligible; for runs that already complete in 2-3
s, n_jobs=-1 may not be worth it.
chunk_size — energy-axis batching
For each k-point the Lopez-Sancho recursion is run as a single batched LAPACK pass over (a chunk of) the energy grid. Larger chunks ⇒ less Python dispatch overhead, more peak memory. The defaults are chosen so memory stays bounded even for very dense energy grids on thick slabs:
Class |
Default chunk |
What it batches |
|---|---|---|
|
|
|
|
|
|
Pass a smaller value (e.g. 32) if you hit memory pressure on a
thick slab, or a larger value if memory is plentiful and you want
every (k, w) pair in one shot.
Choosing what to set
Always set ``n_jobs=-1`` for any nontrivial run. It’s a 3-10× speedup with no precision cost.
Leave ``chunk_size`` alone unless you hit a memory error. Lower it (
chunk_size=32or16) to fix the error.Don’t combine ``n_jobs=-1`` with ``device=”cuda”`` — the GPU is already a batched accelerator; spawning multiple host processes that each grab a CUDA context will fight over the GPU. Use one or the other.
Implementation note
The Lopez-Sancho recursion uses two LAPACK solve calls per
iteration with the same coefficient matrix; these are combined into
a single multi-RHS solve so the LU factorization happens once. That
plus batching across the energy axis accounts for the ~25-35%
speedup at n_jobs=1; the rest of the speedup at n_jobs > 1
comes from real cross-process parallelism on independent k-points.