#1 2022-01-22 21:40:44

VonPire

Member

Registered: 2021-01-14

Posts: 33

Workstation/PC builds for simulations with Salome-Meca/Code_Aster MPI

Hello Everyone,

I decided to open a new topic for open discussion about workstation/PC builds that are used for simulations with Salome-Meca/Code_Aster. This idea came from the fact that I'm searching and buying a new workstation build for Code_Aster MPI simulations.

It would be nice to hear what kind of workstation/PC builds people here have and also any kind of benchmark/performance test comparing between different builds would be welcome.

For example, does the optimal workstation build differ in some way when compared to workstation builds that are used for commercial FEA softwares ?

I think that this kind of discussion would help everyone to build optimal workstations for professional or hobbyist simulations with Salome-Meca/Code_Aster and decrease even more the performance gap to commercial softwares.

This is my current plan for build:

-Processor: AMD Ryzen 9 5950X; 3.4GHz, 16 cores, 32 threads, L2 Cache: 8MB, L3 Cache: 64MB, 24 PCIe 4.0 Lanes, Dual Channel support for RAM.

-Liquid cooling for processor: be quiet! Silent loop 2 360 mm

-RAM: G.Skill Ripjaws V DDR4-3600 CL16 QC – 128GB; DDR4, 3600MHz, Latency: 6-16-16-36, Non-ECC, Intel XMP 2.0

-SSD / Storage cooling: be quiet! MC1 PRO

-Motherboard: ASUS ROG CROSSHAIR VIII DARK HERO, Dynamic OC possibility, 24 PCIe 4.0 lanes.

-SSD storage: WD Black SN850 PCIe 4.0 NVMe M.2 - 1TB; Read Speed: 7000 Mt/s,  Writing Speed: 5300 Mt/s

-Power supply: Corsair RM750x (2021) - 750 Watt

-Case: Corsair 4000D AIRFLOW TG – Black

-GPU: My old PALIT GTX 1060 6GB

This kind of build costs like 2500€ (without GPU price)

What do you guys think ? Would you recommend an Intel 2xCPU or 1xCPU Thread Ripper/EPYC build instead of this ? I wish that 5950X would support PCIe 5.0 and DDR5 and Quad channel.

The new Intel i9-12900K would be tempting, because of the lower price, DDR5 and PCIe5.0 support + faster single and multi core speed at basic CPU benchmarks. This processor has only 24 threads and L3 cache is only 30 MB + the insure and question arises from that this processor have 8p (performance cores) + 8e (smaller and less powerful economical cores) and how this kind of CPU would work with Code_Aster ? 

Feel free to comment on this build and you can also post your workstation builds !

Best Regards,
VonPire

Last edited by VonPire (2022-01-22 21:59:42)

jonas loenartz

Member

Registered: 2021-10-01

Posts: 69

Re: Workstation/PC builds for simulations with Salome-Meca/Code_Aster MPI

Hello,

This built looks pretty good in my opinion, but mainly, if you plan to use it as a PC and occasionally as a workstation for simulations.
Here are some things I think you should consider.

These are mainly consumer PC parts you are listing. This gets especially relevant when it comes to CPU/GPU. The Ryzen 5950x as well as every other consumer model on the market right now only support dual-channel RAM. If you look at the threadripper models by AMD or the Xeon models by Intel, they support 4 or even 8 memory channels. FE-simulations are heavily memory bound, and you will encounter a bottleneck by using dual-channel memory.

I for example have the AMD Ryzen 3950X (16 core), 64Gb DDR4 Ram 3200 MHz (the rest is not that important for this at the moment).

At work I currently use a 5 year old machine with a 28 core Intel Xeon 2,5 GHz processor  and 256Gb of Ram (2666 MHz).
The machine works with 4 RAM channels.
(I don't have all specs in my head, like cache data, but this is more to show you the difference a setup, that is built for this kind of work, can do).

The 5 year old workstation is on average around 30% faster than my PC/workstation at home (same case-setups). In especially memory consuming cases, the gain in speed can even reach around 40-50%.

So my conclusion is, if money isn't an issue and you are planning to use the machine mainly as a workstation for FE-simulations, you should think about going with Threadripper or Xeon models.

If you want a highend PC you use mainly for other stuff like gaming, office-work, pre- and post-processing for FE, etc. and occasionally do hardware-consuming simulations, than the built is a good as it gets I think.
I have no experience with DDR5 memory, but the higher bandwidth could theoretically speed things up quite a bit.

best regards,
Jonas

Last edited by jonas loenartz (2022-01-23 08:58:04)

mf

Member

Registered: 2019-06-18

Posts: 378

Re: Workstation/PC builds for simulations with Salome-Meca/Code_Aster MPI

Hello,

I am also in the process of buying new hardware.
The workstation I use for preparing models is quite old now: HP z420 256Gb RAM, 2687wV2. But still, it runs and runs.
I would then move very big models to a server, the large one I sold yesterday was a HP DL560 G8 4*CPU 4657LV2 768GB RAM. The guy who bought it will do Chia plots on it (20 minutes per plot as opposed to 1 hour on his 9000€ 3995WX 256Gb workstation :-) ). There you have it: Code_Aster and this weird stuff both need only one thing: RAM.

I will probably buy a used HP z8 G4 next week (2 CPUs, RAM I have not decided yet how much, probably 384Gb..). In this chassis+motherboard I can switch to a more modern 2020 CPU next year and also upgrade RAM if needed. And: DDR4 is cheaper now with DDR5 on the horizon.

12900K seems to be very fast but can only address 128GB RAM (it would be for sure good in a workstation+server situation like I had until now. Prepare model on workstation and move it the server for really big simulations). The Xeons of this architecture (Sapphire Rapids) are not out yet. I am sure they will be extremely expensive per CPU (5000€+++) but can use Optane :-) .

I do not see a use case for me now with threadripper (I asked in another thread also), all boards I have seen only can go up to 256Gb RAM. So will go for Xeon again. Apart from Lenovo there are no certified workstations with threadripper and they are very expensive, even used. My main concern is durability and availability of spare parts in 2-3 years plus. A self built computer (also by a company that does such builds!) cannot fulfill that need. I use HP workstations for years now, I am for sure biased. But you can leave them at 100% for weeks without any trouble and I can get spare parts still, for ye old z420.

Mario.

Last edited by mf (2022-01-23 10:03:52)

VonPire

Member

Registered: 2021-01-14

Posts: 33

Re: Workstation/PC builds for simulations with Salome-Meca/Code_Aster MPI

Hello Jonas and Mario,

Thank you for commenting my planned workstation build and telling the experience of your workstations.

This lead to thinking the workstation build again... The serious issue with both 5950x or i9-12900K is indeed the dual channel support resulting to low memory bandwidth. 5950x has 47.68 GiB/s and i9-12900K has 71.53 GiB/s (dual channel, but DDR5 ).

So now I have to think how can I keep the price not going much over 3k€, but have enough memory bandwidth and computing power. This leads to battle of 2 processor candidates:

AMD EPYC 7443P
Year: 03/2021
Cores:24
Threads:48
RAM: DDR4-3200MHz up to 4TB
Memory Bandwidth: Octa channel 190.7 GiB/s
L1 Cache: 1.5 MiB
L2 Cache: 12 MiB
L3 Cache: 128 MiB
Base clock: 2.85 GHz
Turbo clock: 4 GHz
TDP: 200 W
PCIe 4.0 lanes: 128
Price: 1650€

AMD Ryzen Threadripper PRO 3955WX
Year: 07/2020
Cores:16
Threads:32
RAM: DDR4-3200MHz up to 2TB
Memory Bandwidth: Octa channel 190.7 GiB/s
L1 Cache: 1 MiB
L2 Cache: 8 MiB
L3 Cache: 64 MiB
Base clock: 3.9 GHz
Turbo clock: 4.3 GHz
TDP: 280 W
PCIe 4.0 lanes: 128
Price:1200€

The question is that: Is EPYC 7443P better because of the larger L1,L2,L3 caches and additional 8 cores, but lower clock speeds ?
Another question is that could the 7443P be used as workstation build (using Ubuntu 20.04 LTS), because it is intented to be "Server CPU" ?

Best regards,
VonPire

Last edited by VonPire (2022-01-24 11:59:16)

mf

Member

Registered: 2019-06-18

Posts: 378

Re: Workstation/PC builds for simulations with Salome-Meca/Code_Aster MPI

Hello again,

in the last weeks I was going through the data of many CPUs (mostly from the blue company). Modern workstation CPUs (especially the W-3xxx series by Intel) are very much like server CPUs in many regards (many memory channels, many cores, advanced instructions, large cache,.. also large price :-) ) but tuned for single sockets. Turbo frequencies are even higher compared to server CPUs (dedicated WS-CPUs are the fastest CPUs there are). Because in a workstation you'll also need single core performance (many tasks are single core, loading files, etc.).

Regarding your choices I looked at

https://__www.cpubenchmark.net/cpu.php?cpu=AMD+Ryzen+Threadripper+PRO+3955WX&id=3846

and

https://__www.cpubenchmark.net/cpu.php?cpu=AMD+EPYC+7443P&id=4391

I conclude, that 7443P will be faster in all disciplines, interestingly also in single core performance (that's very 'server-CPU-unlike', because single core performance is often not important in servers, but there are exceptions).
The board for 7443P could be more expensive though (maybe a SuperMicro?? very good boards!),

Mario.

Last edited by mf (2022-01-25 13:05:08)

VonPire

Member

Registered: 2021-01-14

Posts: 33

Re: Workstation/PC builds for simulations with Salome-Meca/Code_Aster MPI

Hello,

Thank you Mario for your reply.

I investigated processors from the blue company and AMD and came to conclusion that EPYC 7443P had best performance when considering 1.5k€ to 2k€ processor. So I ordered yesterday EPYC 7443P and ROMED8-2T motherboard. I was lucky and found them with a good price from Poland. I searched over day for them, because they were out of stock nearly every online shop at nordic countries and Germany.

Now I'm thinking about what RAM memory I should choose. EPYC 7443P supports Octa channel memory and DDR4-3200MHz. Capacity would be 258GB so I need 8 memory stick with 32Gb memory. I have understood that the memory for EPYC should be also ECC and registered. So I'm searching DDR4-3200 ECC RDIMM 32Gb memory sticks.

Candidates could be:
Kingston KSM32RD4/32MEI
MICRON MTA18ASF4G72PDZ-3G2E1

However the question arises that should I use Single Rank (1RX4 or 1RX8) or Double Rank (2RX4 or 2RX8) memory sticks to achieve maximum memory bandwidth and latencies or speed ? Do you guys have any knowledge are there any benefit with single rank vs double rank when it comes to RAM with FEA simulations ?

Best Regards,
VonPire

Last edited by VonPire (2022-01-27 18:12:40)

mf

Member

Registered: 2019-06-18

Posts: 378

Re: Workstation/PC builds for simulations with Salome-Meca/Code_Aster MPI

Hello,

this is for sure not easy to find out if SR or DR is quicker. My personal opinion is: I would not worry about it. If you populate every  channel with one DIMM and choose the max. frequency, then you're good to go (never do the following: use less DIMMs than memory channels or populate all DIMM slots. Some boards will lower the frequency if all slots are used, this is my experience with DDR3. This depends on the board, though).
Perhaps you'll find SR vs DR DIMM tests for some DIMMs but likely not for your specific DIMM and likely for gaming applications.
If in doubt, I would choose DR,

Mario.

Last edited by mf (2022-01-28 09:23:48)