TACC Computing

iHARP -> Resources -> TACC Computing
last updated 2023 August 29

TACC Overview      Account Setup       Analysis Portal

Learning Portal   Data Management  Need Help?

Machine Learning on HPC: Introduction to ML Support at TACC

Not sure what TACC is, how it can help with your research or what support is offered.

Click here to view a pdf presentation titled “Machine Learning on HPC: Introduction to ML Support at TACC”

Video Tutorial | Logging in with SSH

Interacting with High-performance computing systems at TACC — New video coming soon

Click here to go directly to Youtube to watch the video

^Top

1. Linux basics: https://wikis.utexas.edu/display/CoreNGSTools/Getting+started+at+TACC#GettingstartedatTACC-Aboutthecommandline
2. Python: https://docs.python.org/3/
3. Pytorch: https://frontera-portal.tacc.utexas.edu/user-guide/ml/
4. Code editors:
   1. Very important for navigating TACC is to master Linux based text editors: (e.g. emacs, nano, gedit, or vi/vim)
   2. Users should go look for instructions for other code editors
5. VS Code instructions on TACC will be issued and posted ASAP

^Top

last updated April 26, 2023

1 – Create a TACC account

All iHARP users need to create a TACC account here: https://portal.tacc.utexas.edu/account-request

2 – Choose a computational resource adequate for your research project needs

The following TACC resources are available to iHARP users:

• Frontera https://frontera-portal.tacc.utexas.edu/user-guide/
• Corral https://portal.tacc.utexas.edu/user-guides/corral

Reading the user guides clarifies the purposes/functionalities and ways of accessing each resource.

3 – Getting assigned to selected TACC resource

To use one or more of these resources users have to be assigned to the iHARP allocations at TACC.

Once you have identified the resource/s that you need, please send in a TACC consulting ticket via    https://portal.tacc.utexas.edu/tacc-consulting/-/consult/tickets/create  with the following information: 

1. Always place the term iHARP in the ticket title/subject line so consultants understand the context of the ticket.
2. In the message, include your TACC username (see step 1)
3. Include which resource you want to be assigned to (see step 2)

Most tickets are resolved within 24 hours. You will then receive an email confirming that you can access the resource you requested.  You can then log in to the TACC user portal https://portal.tacc.utexas.edu/  and review your allocation/s by clicking in the Allocations/Projects and Allocations tab. 

Please, read the user guide corresponding to the resource you plan to use to learn how to get started. 

4 – Basics: TACC Allocations for iHARP

1. Account group for Allocation CCR20014: G-823204, which can access the Frontera machine.

Please, read the user guide corresponding to the resource you plan to use to learn how to get started. 

^Top

^Top

To learn more about how to utilize TACC resources, please check out the learning portal resources. There is a mix of resources available that include webinars, recordings, and additional links.

Portal Link: https://learn.tacc.utexas.edu/
^Top

written by: Sikan Li, Jianwu Wang, Maria Esteva | Last updated April 26 2023

1. iHARP can access the following TACC resources:
   1. Corral Data Server: High-Performance Storage and Data Management, can be directly accessed on Frontera. Copy data from Coral to $WORK or $STOCKYARD on Frontera for group-based experiments and copy results back to Coral for long-term storage.
   2. Frontera GPU CPU hybrid cluster: Largest open-science cluster in the U.S. The allocation is called AI and ML for iHARP, the project charge number is CCR20014 and the UNIX group number is  G-823204.
2. iHarp has a 5 Terabyte allocation on TACC’s high-performance data storage Corral. Everybody in iHARP that is a TACC user and has been assigned to the iHARP Pilot Project can access Corral.
3. Accessing Corral (as well as other TACC machines) is provided via SSH to: data.tacc.utexas.edu.
4. The storage directory for iHARP is: /corral/projects/iHARP-Pilot-project/. Within, each project should have its own directory.
5. Each iHARP research project should name a “data manager” to organize files, device file naming conventions, and manage access policies and permissions within their project directory. The data manager in each group/project should be in charge of making the directory for each project. The first step is to send an email to sli@tacc.utexas.edu with their TACC username and project number to ask for permission to create the project directory.
6. After getting the permission, the data manager can create the directory for their project. The name of the directory should be project-{project number}-{project short title}. Example command to create the project directory: $ mkdir project-5-ice-bed. These directories are meant to store data while the researchers are actively conducting research. These are not published files nor have metadata that explains what the files are.
7. With standard permissions, everybody in iHARP can read data from other projects. However, a best practice is not to read other project’s directory than your own. If you need to access (read/write/execute) another project directory please send an email to the data manager of the specific project requesting permission to do so.
8. The data manager can handle permissions using ACL, to allow/prohibit read/write/execution for other users. See instructions on ACL from the link: https://portal.tacc.utexas.edu/user-guides/corral#managing-files-and-permissions-using-acls
9. To check permission use getfacl command: getfacl myfile, see example output and explanation of output from this command: https://portal.tacc.utexas.edu/tutorials/acls#viewing To check storage quota and usage, go to “Allocations->Projects and Allocations” under  TACC User Portal.
10. Corral is available from other compute resources, but should NOT be used directly in computation. Compute resources available to iHARP users is Frontera  frontera.tacc.utexas.edu.
11. When you need to conduct computation with data that is stored in Corral, the workflow is to copy your data from Corral to the $SCRATCH directory on the compute resource of your choice, conduct your computational work, select the data you need to keep long-term, and then copy it back to Corral. To learn how to copy files with linux command within the same machine, see https://linux.die.net/man/1/cp
12. Data transfer between two Unix/Linux machines can be accomplished using the scp or rsync utility to copy data to and from the login node. Instructions can be found at https://portal.tacc.utexas.edu/user-guides/corral#transferring-cl.
13. Each TACC compute resource has directories to hold data during computation. Table 1 summarizes their usage.

Table 1: TACC File System Usage Recommendations.

More at https://docs.tacc.utexas.edu/tutorials/managingio/

14. All clusters have a local HOME, a shared WORK which is Stockyard, and a local SCRATCH.  Home is small, backed up, and not for data. Work is for data being stored between compute sessions.  Scratch is where HPC jobs run and store files during and between compute sessions.
15. Everyone gets 1 TB of quota but it is not backed up.  It is for keeping things for a while while you work on them.
16. For group-based sharing of compute directories, please check the instructions at https://docs.tacc.utexas.edu/tutorials/sharingprojectfiles/. You might need to change your group of the $STOCKYARD and give it group read/write permission.

Additional references

1. https://docs.tacc.utexas.edu/tutorials/sharingprojectfiles/
2. https://docs.tacc.utexas.edu/tutorials/managingio/https://portal.tacc.utexas.edu/tutorials/managingio#files
3. https://portal.tacc.utexas.edu/user-guides/corral#transferring

^Top

Once you are set up with a TACC user account, if you need further help to identify which resource you need for your project, need help implementing your code in one of the resources, experience roadblocks, require an allocation increase, or to request access to another resource, use the Consulting Ticket  https://tacc.utexas.edu/about/help/

Have an issue? Submit a ticket

Key items to remember:

• Always indicate iHARP at the beginning of the title and describe your issue in detail so that user services can redirect your request to the right expert.
• Submit a ticket for each issue you face when using TACC here: ticket system for TACC issues.
• Provide a detailed explanation of the issue so TACC personnel may quickly understand/reproduce your issue and provide help.
• As stated at https://iharp.umbc.edu/resources/tacc-computing/, always place the term iHARP in the ticket title/subject line so consultants understand the context of the ticket. In the message, include your TACC username (see step 1). Also, include which resource you want to be assigned to (see step 2)
• CC your advisor or project lead as well as to Ms. Sikan Li, sli@tacc.utexas.edu, and Dr. Maria Esteva, maria@tacc.utexas.edu, on the ticket. This will help us track whether issues are solved, how long it takes to solve them and how they were solved.
• If needed, Sikan Li will schedule a time slot during her office hour to discuss and help. An update will be provided to the ticket on the progress after the discussion.
• Feel free to reply to your original ticket to ask about progress if you still do not receive a response after a few workdays.

USE of TACC Analysis Portal https://tap.tacc.utexas.edu/

The TACC analysis portal is an excellent way to use TACCs resources through a graphical user interface and get acquainted with high-performance computing (HPC) resources. Please read the user guide https://docs.tacc.utexas.edu/tutorials/TAP/#tacc-analysis-portal-user-guide. You can start using it once you have completed steps 1, 2, and 3. 

Machine Learning Office Hours with Sikan Li

Sikan Li conducts weekly virtual office hours for consulting related to Machine Learning research projects. Office hours take place every Monday from 1 to 2 pm central and do not require a reservation, just use the Zoom link below.

Link: https://utexas.zoom.us/j/8611857010

Other ways to communicate with TACC experts

Consulting tickets are the best and most efficient ways to communicate with TACC. However, TACC experts that work on the iHARP project are also available on the iHARP slack channel.

^Top

Page created 2022 November 15