Hi Michael Madigan,
Thank you for contacting us with your query.
1. An ssh terminal is preferred over a Jupyter terminal in situations where you need to run a workload that might require long hours to complete. The maximum timeout for the ssh terminal is 24 hours compared to the maximum of 4 hours timeout for a Jupyter notebook. This should not have any impact on the performance otherwise.
Having said that, could you please elaborate on your first question? It seems to be incomplete.
2. Uploading a .csv file to the notebook depends a lot on the speed offered by your Internet connection.
You can try doing a secure copy (scp) using WinSCP in Windows and scp in Linux.
Also, could you please give us the size of the .csv files that you are trying to upload?
Please let us know if the solutions provided here were helpful.
1. My apologies. It meant to say something along the lines of... "Any recommendations on ways to increase compute power or resource allocation while using the Jupyter notebooks?"
2. I do nothing speed of the internet is the issue here, as my internet is currently listed as having a 289 Mbit/s link speed. I am unable to ssh into the cluster or any external environment from my local machine due to a company firewall. As for the size, they are not small files... I was working on loading a 10GB .csv file, but we were hoping to upload much larger files than that in the near future.
Thank you for the help.
1. I could suggest optimizations specific to the framework. This would go both for ssh terminal and Jupyter terminal -
Please follow the given link for optimizations - https://software.intel.com/en-us/articles/tips-to-improve-performance-for-popular-deep-learning-frameworks-on-multi-core-cpus
From the link, you could try a few variations as well such as setting KMP_BLOCK_TIME to 30 or 0 and NUM_PARALLEL_EXEC_UNITS to 6 or 12
2. We will check and get back to you if there are better ways to perform file uploads. In the meanwhile, could you please be a bit more specific on how much time it took for you to upload the .csv, the size of the file and the time taken to upload it.
I hope the Intel SME interaction has helped with the issue.
We will be closing the thread now. Please open another thread in case you are facing any other issue.