Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
en:services:application_services:high_performance_computing:transfer_data [2019/12/10 12:03]
hsommer [ownCloud]
en:services:application_services:high_performance_computing:transfer_data [2020/01/29 16:29] (current)
mboden [Transfering Large Quantaties of Data]
Line 10: Line 10:
  
 For the filesystem /scratch2 we don't have a dedicated transfernode. For this filesystem you need to use the frontend ''​gwdu103.gwdg.de''​ (instead of ''​transfer-scc.gwdg.de''​). For the filesystem /scratch2 we don't have a dedicated transfernode. For this filesystem you need to use the frontend ''​gwdu103.gwdg.de''​ (instead of ''​transfer-scc.gwdg.de''​).
 +
 +=== Transfering Large Quantaties of Data ===
 +
 +If you need to transfer large quantities of data to our system, it may be advisable to copy directly to /scratch to increase the transfer rate, since the write-performance of our home file systems can sometimes be the limiting factor. Transfering to /scratch directly will not have this limitation.
 =====  Within the GÖNET ​ ===== =====  Within the GÖNET ​ =====
  
Line 18: Line 22:
 **/​scratch** **/​scratch**
  
-using ssh (scp or rsync tunneled through ssh). Here is an example for copying data from your computer to scratch (please replace {SRC-DIR},​{DST-DIR} and {USERID}):+using ssh (scp or rsync tunneled through ssh). Here is an example for copying data from your computer to scratch (please replace ​''​{SRC-DIR}''​''​{DST-DIR}'' ​and ''​{USERID}''​):
 <​code>​ <​code>​
 scp -rp {SRC-DIR} {USERID}@transfer-scc.gwdg.de:/​scratch/​{USERID}/​{DST-DIR} scp -rp {SRC-DIR} {USERID}@transfer-scc.gwdg.de:/​scratch/​{USERID}/​{DST-DIR}
Line 41: Line 45:
 =====  Outside of the GÖNET (from the Internet) ​ ===== =====  Outside of the GÖNET (from the Internet) ​ =====
  
-If your computer is located //outside the GÖNET//, you can use the server ''​transfer-scc.gwdg.de''​ to copy data from and to:+If your computer is located //outside the GÖNET//, you can use the server ''​transfer.gwdg.de''​ to copy data from and to:
  
 **your Home directory** **your Home directory**
Line 48: Line 52:
  
 For security reasons this node is not located inside the SCC network and therefor only has your Home directory but not /scratch accessible. For security reasons this node is not located inside the SCC network and therefor only has your Home directory but not /scratch accessible.
 +
 +====  Getting Access to Scratch from Outside the GÖNET ​ via VPN (recommended) ====
 +
 +The easiest way to get access to /scratch is to use our [[en:​services:​network_services:​vpn:​start|VPN]]. After activating the VPN connection as described in the documentation and you will be inside the GÖNET and therefore be able to use transfer-scc.gwdg.de as described above.
  
 ====  Getting Access to Scratch from Outside the GÖNET ​ (ssh proxy method)==== ====  Getting Access to Scratch from Outside the GÖNET ​ (ssh proxy method)====