I wonder if you can help me to build the perfect PC setup for the following task:
I am going to perform daily financial market analysis on multiple instruments using Excel (large workbooks around 300 and 500 mbs), Matlab and Automation software. The price will be loaded into Excel, the results would be fed into Matlab to create charts (this sequence will be repeated many times). I plan to sell the resultant analysis at my website for a set monthly subscription – so this can be considered a production workstation. The total process would take 5-7 hours daily. I need this process to run as error-free as possible – absolutely predictably on autopilot. So I am not planning to over-clock the CPU.
I am deciding between a workstation built on Intel Core i7-3930K or the one built on single Intel Xeon E5-2630. These CPUs are roughly the same in price (same number of cores, different speed though) with the I7 being much faster one. But I am more concerned with reliability and stability of this setup. Do you think the ECC memory can help eliminate system crashes when the analysis job is running? I need to be able to connect this PC remotely to initiate the analysis jobs as well. I am also thinking of Intel Core i7-3930K which is very fast but not sure about its stability for long-duration number crunching sessions (it can overheat if run at full speed for many hours?).
Please let me know what you think,
Dave
Thread Recap (last 10 posts from newest to oldest)
Mar 25, 13 at 5:47am
dave777
I just wanted to let you know that I recently bought the HP Z420 machine with the e5-1650 processor and 8 gb ecc ram!) runs great!) thank you for your comments and directions!!!!))))
i had one more question - I do need to think fo a backup solution just in case - I was wondering if you can recommend anything for this - do you think there is a way to upload and run my own 64-bit vmware image containing the 64 bit office and Matlab to some private cloud service??? just to be sure I have another place to recalculate all stuff for emergency cases...would it be possible to upload such a wmware image to some cloud service? what do you think? thanks!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Mar 10, 13 at 8:42pm
Crazy Engineer
If you are stuck in Excel and processing is taking so long, I would imagine that you are doing some kind of regression analysis. If so take a look at a company called Parabon Computation and and piece of software they provide called Crush. It is an add-in to Excel that will let you take the work you have and running on their Gird Service. That is the cheapest and easiest way to run it quickly and completely. No re-coding of you work, just re-targets it to the number of processors you specify 1, 10 , or 1000's). The second option, assuming you are using MatLab to do the analaysis is to look at GPU based running of the functions. The latest release is highly capable, and you get essentially add 1000 cores to your process vary inexpensively ( i.e. get the product to market faster). It all depends on whether you have the understanding of the programming of MatLab.
Google Matlab and GPU processing and you can see if the functions you are using are optimized for GPU processing.
A program that can use multiple threads will definitely be as many times better, as many more cores you have.
1 core used and 5 wasted is obviously a waste if you could have 6 of 6 in use. Virtual cores make little difference.
I don't have Excel, so I'm just speculating.
And as for the difficult to replace memory. If it's that important, I would buy 1 extra at some point, so I could replace it with in hours of the failure.
thank you very much for your reply)) I think I have finally nailed a very attarctive offer that might well suit my needs perfectly....well nearly perfectly.... I have found a workstation by HP whichh features the much coveted Intel Xeon E5-1650 processor!!!!!!! and its price is within the constraints
I am slightly weary of building my own custom build so this one looks very attarctive....it features the e5-1650 as well as the 8 gbs of ram
and is all factory packed and of course ecc and everythign super fail prooof I guess...please let me know what you think!!!!!)
I have a couple of more questions...do you think it mitgh be possibel to add more 8GBs of Ram and a video card without voiding the warranty? thanks!! feel so close to the final step!)
Dec 07, 12 at 1:09pm
Randome
A program that can use multiple threads will definitely be as many times better, as many more cores you have.
1 core used and 5 wasted is obviously a waste if you could have 6 of 6 in use. Virtual cores make little difference.
I don't have Excel, so I'm just speculating.
And as for the difficult to replace memory. If it's that important, I would buy 1 extra at some point, so I could replace it with in hours of the failure.
Dec 05, 12 at 2:27pm
dave777
I wonder if any one with i7 3930K or XEON E3 127V2 can comment on the multi-threading boost to EXCEL 2010 or EXCEL 2007 performance???? I need this to make the final decision between these two setups...do you think the multi-threaded EXCEL 2010 in the case of i7 3930K or XEON E3 127V2 can be significantly faster than the single threaded EXCEL 2003? thanks a lot!)
Nov 29, 12 at 10:19am
dave777
thank you for your generous attention!)
I need to introduce one more element into the equation...sorry to have left it out before....I need to spend quite some time outside of the Europe and USA - in eth Western Europe - where the server grade parts take a week to arrive - a week is intolerable time to wait for any part in case the setup fails....I will be providing the analysis each day and it needs to be updated on a daily basis...new signals can orccur any time...
for a server-based example, I have been recommended the followign setup
the cpu and motehrboard are more or less easily available here but the RAM 16GB DDR3 1600 ECC Unbuff which is only accepted by this setup needs to be ordered and will take from 7 to 10 days to arrive.....I have not foudn otehr unuffered ECC on teh market now...
I mean if any part of this seup fails - Especially memory - I will need to replace it...but only within one weeks time....
if i go with the i7 3930k based setup - I shoudl be able to get any kind of the memory in no time at all from teh local PC shops...The XEON+ECC+SERVER mobo looks very reliable but if anythign beraks down I stand the chance of replacing the broken part much faster with the 3930k based setup....considering the power of 3930k I mitgh be able to crunch all data in a matter of 5 o6 hours so no need for 24/7 operation for now....
I will be doign maintenance myself at first - do you think a server-based setup mitghbe harder to maintain than a desctop based setup?
I wonder if you have tried running your i73930k for long periods of time..like 10 hours at the maximum load....I wonder how it behaved? lets say for 8-10 hours in a row....at 95% (no over-clocking) - did you see any hints of instability in its performance? Thanks!
CPU's are actually a lot more capable than you think, I had a i7 950 setup (Gen 1) that I ran 24/7 under full load, while overclocked to 4ghz, using folding @ home, only time it wasn't at full load was when the power went out, this was used for 2 years, until I upgraded this week. Same story with GPU's as I have had my 5830 running at constant full load 24/7 while mining bitcoins (ie. lots and lots of calculations) for over a year.
as long as the CPU has proper cooling... it can almost run forever at full load.
I wonder if you have tried running your i73930k for long periods of time..like 10 hours at the maximum load....I wonder how it behaved? lets say for 8-10 hours in a row....at 95% (no over-clocking) - did you see any hints of instability in its performance? Thanks!
I have an i7 2600k, the earlier generation, and I've ran it at close to max load for around 5 hours with no problems (it was a high end game). I've also had this computer run for up to a week before I started getting errors. This is with the CPU underclocked though, since it is really hot where I am so I'd rather it produce less heat.
Nov 28, 12 at 2:33pm
dave777
I wonder if you have tried running your i73930k for long periods of time..like 10 hours at the maximum load....I wonder how it behaved? lets say for 8-10 hours in a row....at 95% (no over-clocking) - did you see any hints of instability in its performance? Thanks!
server hardware is purpose built for what youre looking for, that is, sustained, uninterrupted, reliable operation.
ECC memory is also a bit slower than non-ECC memory, but for sustained operation, its your guy.
as for multicore performance, heres some info concerning excel 2007 to present:
"Most workbooks show a significant improvement in calculation speed on a system with multiple cores. The degree of improvement depends on how many independent calculation trees the workbook contains. If you make a workbook that contains one continuous chain of formulas, it will not show any multithreaded calculation (MTC) performance gain, whereas a workbook that contains several independent chains of formulas will show gains close to the number of processors available."
"Results showed calculation times ranging from no improvement to better than theoretical (2x/4x) improvement on both the dual-core and quad-core systems. Typical (median) improvement for a system with an Intel dual-core Pentium 4 at 3.0 GHz with 1 GB of RAM compared to the same file calculating in Excel 2003 was 48 percent, or a 1.92x speedup. Typical (median) speedup for a system with an Intel quad-core Xeon at 3.0 GHz with 4 GB of RAM was 76 percent, or a 4.17x speedup. Similar speed improvements were observed on other processors and platforms. Improvements beyond theoretical speedup (because of multithreading) are attributed to other performance improvements starting in Excel 2007, such as improvements to the speed of function execution."
looks like if the type of calculation youre doing is supported with multicore processing, youll get great performance gains. its scalable, so the more cores you have, the better performance you should see.
some forms of calculating are not supported by multithreading / multicores however.
"Some Excel features do not use multithreaded calculation. For example: Data table calculation (however, structured references to tables do use MTC). User-defined functions (however, XLL functions can be multithread-enabled). XLM functions. INDIRECT, CELL functions that use either the format2 or address options. GETPIVOTDATA and other functions referring to PivotTables or cubes. Range.Calculate and Range.CalculateRowMajorOrder. Cells in circular reference loops."
yes thanks for this helpful answer...I do have formula streams that are independent of one another but which converge at the final value - do you think this can be considered a multi-=threaded ready setup? thanks a lot!)
Powered by neoforums v1.6.5b (aperture) Copyright Neo Era Media, Inc. 1999-2013
