User Tools

You are here: Welcome to ITRSS Wiki » ITRSS UM public wiki page » Getting Started with HPC » Hellbender

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
pub:hpc:hellbender [2025/01/16 21:17] – [Globus] bjmfg8pub:hpc:hellbender [2025/03/06 20:00] (current) – robertschrist
Line 140: Line 140:
 **What is the Difference between High Performance and General Performance Storage?** **What is the Difference between High Performance and General Performance Storage?**
  
-On Pixstor, which is used for standard HPC allocations, general storage is pinned to the SAS disk pool while high performance allocations are pinned to all flash NVME pool.  Meaning writes and recent reads will have lower latency with HPC allocations.+On Pixstor, which is used for standard HPC allocations, general storage is pinned to the SAS disk pool while high performance allocations are pinned to all flash NVME pool.  Meaning writes and recent reads will have lower latency with High performance allocations.
    
 On VAST, which is used for non HPC and mixed HPC / SMB workloads, the disks are all flash but general storage allocations have a QOS policy attached that limits IOPS to prevent the share from the possibility of saturating the disk pool to the point where high-performance allocations are impacted.  High Performance allocations may also have a QOS policy that allows for much higher IO and IOPS.  RSS reserves the right to move general store allocations to lower tier storage in the future if facing capacity constraints. On VAST, which is used for non HPC and mixed HPC / SMB workloads, the disks are all flash but general storage allocations have a QOS policy attached that limits IOPS to prevent the share from the possibility of saturating the disk pool to the point where high-performance allocations are impacted.  High Performance allocations may also have a QOS policy that allows for much higher IO and IOPS.  RSS reserves the right to move general store allocations to lower tier storage in the future if facing capacity constraints.
Line 153: Line 153:
   * Workloads that require sustained use of low latency read and write IO with multiple GB/s, generally generated from jobs utilizing multiple NFS mounts   * Workloads that require sustained use of low latency read and write IO with multiple GB/s, generally generated from jobs utilizing multiple NFS mounts
  
 +
 +**Snapshots**
 +
 +  *VAST default policy retains 7 daily and 4 weekly snapshots for each share
 +  *Pixstor default policy is 10 daily snapshots
  
 **__None of the cluster attached storage available to users is backed up in any way by us__**, this means that if you delete something and don't have a copy somewhere else, it is gone. Please note the data stored on cluster attached storage is limited to Data Class 1 and 2 as defined by [[https://www.umsystem.edu/ums/is/infosec/classification-definitions| UM System DCL]]. If you have need to store things in DCL3 or DCL4 please contact us so we may find a solution for you. **__None of the cluster attached storage available to users is backed up in any way by us__**, this means that if you delete something and don't have a copy somewhere else, it is gone. Please note the data stored on cluster attached storage is limited to Data Class 1 and 2 as defined by [[https://www.umsystem.edu/ums/is/infosec/classification-definitions| UM System DCL]]. If you have need to store things in DCL3 or DCL4 please contact us so we may find a solution for you.
Line 221: Line 226:
  
   * **[[https://status.missouri.edu| UM System Status Page]]**   * **[[https://status.missouri.edu| UM System Status Page]]**
-  * **[[https://po.missouri.edu/scripts/wa.exe?SUBED1=RSSHPC-L&A=1| RSS Announcement List: Please Sign Up]]**+  * **[[https://LISTS.UMSYSTEM.EDU/scripts/wa-UMS.exe?SUBED1=RSSHPC-L&A=1&SUB=1| RSS Announcement List: Please Sign Up]]**
   * **[[https://missouri.qualtrics.com/jfe/form/SV_6zkkwGYn0MGvMyO|RSS Services: Order Form]]**   * **[[https://missouri.qualtrics.com/jfe/form/SV_6zkkwGYn0MGvMyO|RSS Services: Order Form]]**
   * **[[https://request.itrss.umsystem.edu/|Hellbender: Account Request Form]]**   * **[[https://request.itrss.umsystem.edu/|Hellbender: Account Request Form]]**
Line 353: Line 358:
 ==== Open OnDemand ==== ==== Open OnDemand ====
  
-  * https://ondemand.rnet.missouri.edu - Hellbender Open OnDemand (HB OOD) +  * https://ondemand.rnet.missouri.edu - Hellbender Open OnDemand (Researcher) 
-  * https://hb-classes.missouri.edu - Hellbender Classes Open OnDemand (HB OOD classes)+  * https://hb-classes.missouri.edu - Hellbender Classes Open OnDemand (Classes)
  
 OnDemand provides an integrated, single access point for all of your HPC resources. The following apps are currently available on Hellbender's Open Ondemand. OnDemand provides an integrated, single access point for all of your HPC resources. The following apps are currently available on Hellbender's Open Ondemand.
Line 873: Line 878:
 **Endpoint** **Endpoint**
   * An endpoint in Globus is a network location (like a server, a cluster, or a personal computer) that has been configured to use Globus services. It is an access point for data transfer, allowing users to move or share data between their endpoint and others.   * An endpoint in Globus is a network location (like a server, a cluster, or a personal computer) that has been configured to use Globus services. It is an access point for data transfer, allowing users to move or share data between their endpoint and others.
-  * **The Hellbender/RDE endpoint “U MO ITRSS RDE”. If you are trying to transfer files from your university OneDrive select “U MO ITRSS RDE - M365”**+  * **The Hellbender/RDE endpoint is: “U MO ITRSS RDE”.** 
 +  * **If you are trying to transfer files from your university OneDrive select “U MO ITRSS RDE - M365”**
  
 **Data Transfer Node (DTN)** **Data Transfer Node (DTN)**
Line 899: Line 905:
   * If you’ve already used another account with Globus, you can choose “Link to an existing account.”   * If you’ve already used another account with Globus, you can choose “Link to an existing account.”
   * You may be prompted to provide additional information such as your organization and whether or not Globus will be used for commercial purposes. Complete the form and click "Continue."   * You may be prompted to provide additional information such as your organization and whether or not Globus will be used for commercial purposes. Complete the form and click "Continue."
-{{:pub:hpc:welcome_-_successfully_logged_in_-_5.png?600|}}+{{:pub:hpc:welcome_-_successfully_logged_in_-_5.png?1200|}}
  
 Finally, you need to give Globus permission to use your identity to access information and perform actions (like file transfers) on your behalf. Finally, you need to give Globus permission to use your identity to access information and perform actions (like file transfers) on your behalf.
-{{:pub:hpc:globus_terms_6.png?400|}} {{:pub:hpc:globus_allow_or_deny_6.png?800|}}+{{:pub:hpc:globus_terms_6.png?600|}} {{:pub:hpc:globus_allow_or_deny_6.png?800|}}
  
 ==== Tutorial: Globus File Manager ==== ==== Tutorial: Globus File Manager ====
  
 After you’ve signed up and logged in to Globus, you’ll begin at the File Manager. After you’ve signed up and logged in to Globus, you’ll begin at the File Manager.
 +
 +**note:  Symlinks may not be transferred via Globus, preview:
 +https://docs.globus.org/faq/transfer-sharing/#how_does_globus_handle_symlinks
 +If symlinks need to be copied, consider using the rsync on the DTN with with the -l flag**
 +
 +
 +
  
 The first time you use the File Manager, all fields will be blank: The first time you use the File Manager, all fields will be blank:
  
  
-{{:pub:hpc:file_manager_homepage.png?800|}}+{{:pub:hpc:file_manager_homepage.png?1200|}}
  
 **Key Concept: Collection** **Key Concept: Collection**
Line 923: Line 936:
  
 **Access A Collection** **Access A Collection**
-  * Click in the Collection field at the top of the File Manager page and type "globus tutorial endpoint".+  * Click in the Collection field at the top of the File Manager page and type "globus tutorial collection 1".
   * Globus will list collections with matching names. The collections Globus Tutorial Endpoint 1 and Globus Tutorial Endpoint 2 are collections administered by the Globus team for demonstration purposes and are accessible to all Globus users without further authentication.   * Globus will list collections with matching names. The collections Globus Tutorial Endpoint 1 and Globus Tutorial Endpoint 2 are collections administered by the Globus team for demonstration purposes and are accessible to all Globus users without further authentication.
  
-{{:pub:hpc:globus_endpoint_tutorial_search.png?600|}}+{{:pub:hpc:collection_search.png?800|}}
  
-  * Click on Globus Tutorial Endpoint 1.  +  * Click on Globus Tutorial Collection 1.  
-  * Globus will connect to the collection and display the default directory, /~/. (It will be empty.) Click the "Path" field and change it to /share/godata/. Globus will show the files in the new path: three small text files+  * Globus will connect to the collection and display the default directory, /~/. (It will be empty.) Click the "Path" field and change it to home/share/godata/. Globus will show the files in the new path: three small text files
  
- +{{:pub:hpc:test_collection_godata1.png?800|}}
-{{:pub:hpc:tutorial_1.png?600|}} +
- +
-{{:pub:hpc:tutorial_2.png?600|}}+
  
 **Request A File Transfer** **Request A File Transfer**
Line 940: Line 950:
   * A new collection panel will open, with a "Transfer or Sync to" field at the top of the panel.   * A new collection panel will open, with a "Transfer or Sync to" field at the top of the panel.
  
-{{:pub:hpc:tutorial_3.png?600|}}+{{:pub:hpc:transfer_or_sync.png?1200|}}
  
-  * Find the Globus Tutorial Endpoint 2 collection and connect to it as you did with the Globus Tutorial Endpoint 1 above. +  * Find the "Globus Tutorial Collection 2" collection and connect to it as you did with the Globus Tutorial Endpoint 1 above. 
   * The default directory, /~/ will again be empty. Your goal is to transfer the sample files here.   * The default directory, /~/ will again be empty. Your goal is to transfer the sample files here.
-  * Click on the left collection, Globus Tutorial Endpoint 1, and select all three files. The Start> button at the bottom of the panel will activate.+  * Click on the left collection, Globus Tutorial Collection 1, and select all three files. The Start> button at the bottom of the panel will activate.
    
-{{:pub:hpc:tutorial_4.png?600|}}+{{:pub:hpc:select_files_start.png?800|}}
  
   * Between the two Start buttons at the bottom of the page, the Transfer & Sync Options tab provides access to several options.   * Between the two Start buttons at the bottom of the page, the Transfer & Sync Options tab provides access to several options.
Line 953: Line 963:
   * Click the Start> button to transfer the selected files to the collection in the right panel. Globus will display a green notification panel—​confirming that the transfer request was submitted—​and add a badge to the Activity item in the command menu on the left of the page.   * Click the Start> button to transfer the selected files to the collection in the right panel. Globus will display a green notification panel—​confirming that the transfer request was submitted—​and add a badge to the Activity item in the command menu on the left of the page.
  
-{{:pub:hpc:tutorial_5.png?600|}} +{{:pub:hpc:transfer_success_submitted.png?800|}}
- +
-{{:pub:hpc:tutorial_6.png?600|}}+
  
 **Confirm Transfer Completion** **Confirm Transfer Completion**
Line 963: Line 971:
   * On the Activity page, click the arrow icon on the right to view details about the transfer. You will also receive an email with the transfer details.   * On the Activity page, click the arrow icon on the right to view details about the transfer. You will also receive an email with the transfer details.
  
-{{:pub:hpc:tutorial_7.png?600|}}+{{:pub:hpc:transfer_complete.png?1200|}}
  
 ==== Tutorial: Sharing Data - Create a Guest Collection ==== ==== Tutorial: Sharing Data - Create a Guest Collection ====
Line 977: Line 985:
   * Select the collection that has the files/folders you wish to share and, if necessary, activate the collection.   * Select the collection that has the files/folders you wish to share and, if necessary, activate the collection.
   * Highlight the folder that you would like to share and Click Share in the right command pane. *Note: Sharing is available for folders. Individual files can only be shared by sharing the folder that contains them.   * Highlight the folder that you would like to share and Click Share in the right command pane. *Note: Sharing is available for folders. Individual files can only be shared by sharing the folder that contains them.
-{{:pub:hpc:guestcollection1.png?600|}}+{{:pub:hpc:guestcollection1.png?1200|}}
  
 Provide a name for the guest collection and click “Add Guest Collection”. Provide a name for the guest collection and click “Add Guest Collection”.
 If this is the first time you are accessing the collection, you may need to authenticate and consent to allow Globus services to manage your collections on your behalf. If this is the first time you are accessing the collection, you may need to authenticate and consent to allow Globus services to manage your collections on your behalf.
  
-{{:pub:hpc:guestcollection2.png?600|}}+{{:pub:hpc:guestcollection2.png?1200|}}
  
-{{:pub:hpc:guestcollection3.png?600|}}+{{:pub:hpc:guestcollection3.png?1200|}}
  
 When your collection is created, you’ll be taken to the Sharing tab, where you can set permissions. When your collection is created, you’ll be taken to the Sharing tab, where you can set permissions.
 As shown below, the starting permissions give read and write access (and the Administrator role) to the person who created the collection. As shown below, the starting permissions give read and write access (and the Administrator role) to the person who created the collection.
  
-{{:pub:hpc:guestcollection5.png?600|}}+{{:pub:hpc:guestcollection5.png?1200|}}
  
   * Click the Add Permissions button or icon to share access with others.   * Click the Add Permissions button or icon to share access with others.
Line 996: Line 1004:
   * Globus will display matching identities. Pick from the list. If the user hasn’t used Globus before or you only have an email address, enter the email address and click Add.   * Globus will display matching identities. Pick from the list. If the user hasn’t used Globus before or you only have an email address, enter the email address and click Add.
   * This example grants read access to Globus user wassmanr:   * This example grants read access to Globus user wassmanr:
-{{:pub:hpc:guestcollection7.png?600|}}+{{:pub:hpc:guestcollection7.png?1200|}}
  
  

Page Tools

IT Research Support Solutions Wiki

Site Tools