Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
85 commits
Select commit Hold shift + click to select a range
8ee3fce
Fix typo in title from 'Publciations' to 'Publications'
Amanda-dong Jul 6, 2026
2e7b1b6
Fix formatting in run_hf_model.md
Amanda-dong Jul 6, 2026
be95f45
Update 10_spec_sheet.md
Amanda-dong Jul 6, 2026
a347b69
Update storage-finder-data.mdx
Amanda-dong Jul 6, 2026
31fd9ca
Fix typo in permission settings section
Amanda-dong Jul 6, 2026
eb79b2e
Fix typo in onboarding documentation
Amanda-dong Jul 6, 2026
6dfd18c
Update 01_temperature.md
Amanda-dong Jul 6, 2026
c45ab1e
Update 06_large_number_of_small_files.md
Amanda-dong Jul 14, 2026
204c2d7
Update 03_globus.md
Amanda-dong Jul 14, 2026
79fa339
Update 07_transferring_cloud_storage_data_with_rclone.md
Amanda-dong Jul 14, 2026
8b04527
Update 02_data_transfers.md
Amanda-dong Jul 14, 2026
a1f1891
Update 04_walkthrough_renew_hpc_account.md
Amanda-dong Jul 14, 2026
9b0d4f7
Update 01_intro.md
Amanda-dong Jul 14, 2026
dda0fb9
Update 02_run_hf_model.md
Amanda-dong Jul 14, 2026
a56f4fd
Update 03_vLLM.md
Amanda-dong Jul 14, 2026
c61aae7
Update 03_pytorch_dpp.md
Amanda-dong Jul 14, 2026
076ee1e
Update 02_pytorch_intro.md
Amanda-dong Jul 14, 2026
2f44cc3
Update 02_slurm_main_commands.md
Amanda-dong Jul 14, 2026
4e7b7aa
Update 01_slurm_submitting_jobs.md
Amanda-dong Jul 14, 2026
9ceaebb
Update 03_singularity_with_conda.md
Amanda-dong Jul 14, 2026
f1f01d4
Update 02_data_transfers.md
Amanda-dong Jul 22, 2026
c3290e8
Update 06_large_number_of_small_files.md
Amanda-dong Jul 22, 2026
c73c55b
Refactor rclone documentation for clarity and consistency
Amanda-dong Jul 22, 2026
013d6e0
Update 01_intro.mdx
Amanda-dong Jul 22, 2026
8c9bc48
Update 01_intro.md
Amanda-dong Jul 22, 2026
dba7409
Fix capitalization in eligibility section header
Amanda-dong Jul 22, 2026
8ec0022
Update 03_quickstart.mdx
Amanda-dong Jul 22, 2026
feaa85c
Update 01_intro.md
Amanda-dong Jul 22, 2026
7f10d82
Update 02_nih_strides.md
Amanda-dong Jul 22, 2026
55c31e7
Update section title for workspace access
Amanda-dong Jul 22, 2026
31b3456
Update 03_retrieval_augmented_generation.mdx
Amanda-dong Jul 22, 2026
d3990dd
Update 02_embeddings.mdx
Amanda-dong Jul 22, 2026
21e64b6
Fix typos and formatting in embeddings guide
Amanda-dong Jul 22, 2026
5dfd9ae
Update 02_data_management.md
Amanda-dong Jul 22, 2026
4ad397d
Update 02_walkthrough_request_hpc_account.mdx
Amanda-dong Jul 22, 2026
f35d0b2
Update 01_intro_slurm_accounts.mdx
Amanda-dong Jul 22, 2026
7f3d868
Correct capitalization in 'Fine Tuning' header
Amanda-dong Jul 22, 2026
3b1fa96
Update headings for consistency in batch inference guide
Amanda-dong Jul 22, 2026
5f1c2d4
Update 04_requesting_an_allocation.mdx
Amanda-dong Jul 22, 2026
361a43a
Update 01_getting_and_renewing_an_account.mdx
Amanda-dong Jul 22, 2026
8aa6a80
Update 01_connecting_to_hpc.mdx
Amanda-dong Jul 22, 2026
7f23460
Update 05_best_practices.mdx
Amanda-dong Jul 22, 2026
a87a6cb
Update 08_sharing_data_on_hpc.mdx
Amanda-dong Jul 22, 2026
3e702cf
Update 01_intro.md
Amanda-dong Jul 22, 2026
271b615
Update 04_python_packages_with_virtual_environments.mdx
Amanda-dong Jul 22, 2026
65def1b
Update 07_sqlite_handling_large_structured_data.md
Amanda-dong Jul 22, 2026
035bd06
Update 05_r_packages_with_renv.mdx
Amanda-dong Jul 22, 2026
c21f0fc
Update 02_licensed_software.md
Amanda-dong Jul 22, 2026
4d8a3da
Correct header casing for Apptainer section
Amanda-dong Jul 22, 2026
343eed3
Update 03_singularity_with_conda.md
Amanda-dong Jul 22, 2026
7e46f01
Update 03_pytorch_dpp.md
Amanda-dong Jul 22, 2026
604c5db
Update 06_conda_environments.mdx
Amanda-dong Jul 22, 2026
c0a5db0
Update 01_ood_intro.mdx
Amanda-dong Jul 22, 2026
dc18d45
Update 03_vLLM.md
Amanda-dong Jul 22, 2026
c14816d
Fix typos and improve formatting in Desktop.md
Amanda-dong Jul 22, 2026
8e51710
Update 03_matlab_proxy.mdx
Amanda-dong Jul 22, 2026
a5901eb
Update 05_RStudio.mdx
Amanda-dong Jul 22, 2026
7499032
Update section header for Singularity container
Amanda-dong Jul 22, 2026
e74a371
Correct section header from 'kernel' to 'Kernel'
Amanda-dong Jul 22, 2026
1d85eb9
Update 07_igv.mdx
Amanda-dong Jul 22, 2026
0b60af3
Update 01_intro.mdx
Amanda-dong Jul 22, 2026
2595e8f
Update 03_moving_looking.mdx
Amanda-dong Jul 22, 2026
e96a31a
Update 06_jbrowse_genome_browser.mdx
Amanda-dong Jul 22, 2026
37fa18f
Update 04_writing_reading_files.mdx
Amanda-dong Jul 22, 2026
88e1d6c
Update 05_wildcards_pipes.mdx
Amanda-dong Jul 22, 2026
7fbd786
Update 02_why_use_cluster.mdx
Amanda-dong Jul 22, 2026
4161732
Fix typo in HPC tutorial regarding storage locations
Amanda-dong Jul 22, 2026
696bbbb
Update 06_scripts_variables_loops.mdx
Amanda-dong Jul 22, 2026
da88506
Update 06_modules.mdx
Amanda-dong Jul 22, 2026
e981e9a
Update 04_scheduler_fundamentals.mdx
Amanda-dong Jul 22, 2026
b1caeca
Update 05_environment_variables.mdx
Amanda-dong Jul 22, 2026
0f0b1d1
Update section title to 'Using Resources Effectively'
Amanda-dong Jul 22, 2026
e3cebb5
Update 02_connecting_to_hpc.mdx
Amanda-dong Jul 22, 2026
30558de
Update 07_transferring_files_remote.mdx
Amanda-dong Jul 22, 2026
85dd830
Update header to capitalize 'Parallel' in tutorial
Amanda-dong Jul 22, 2026
448e5b5
Update 10_using_resources_responsibly.mdx
Amanda-dong Jul 22, 2026
0d19636
Update header capitalization in intro.md
Amanda-dong Jul 22, 2026
c9a60fe
Update 02_env_roles.mdx
Amanda-dong Jul 22, 2026
ac8b2cc
Revise FAQ section headings to title case
Amanda-dong Jul 22, 2026
5cb34be
Update 01_basics.mdx
Amanda-dong Jul 22, 2026
571f82e
Update header to use title case for consistency
Amanda-dong Jul 22, 2026
9c9dabc
Update README.md
Amanda-dong Jul 22, 2026
117e5cd
Merge branch 'main' into Typo-correction-by-Harper
Amanda-dong Jul 22, 2026
a3ff75b
Update 06_modules.mdx
Amanda-dong Jul 22, 2026
bdd2d7c
Update 08_running_parallel_job.mdx
Amanda-dong Jul 22, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions docs/cloud/02_gcp_self_managed/01_intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ We facilitate access to Google Cloud Platform (GCP) for your research projects.

In addition to the I2 Net+ GCP benefits, NYU scholars can enjoy significant discounts in using GCP resources in their research project through a 3 year commitment (started in 2019) NYU made in GCP. The NYU network connects with GCP via Partner Interconnect using [Internet2 Cloud Connect (I2CC)][internet2-cc] service.

## Why work with the NYU Research Cloud team to deploy your research project on GCP?
## Why Work with the NYU Research Cloud Team to Deploy Your Research Project on GCP?

NYU researchers who work with the Research Cloud team to deploy projects on GCP may benefit from the following:
- Discounted rates, lower GCP project cost: Through NYU's participation in the Internet2 Net+ agreement, as well as the 3-year commitment NYU made to using GCP, GCP projects enjoy discounted rates, lowering the cost of the project. The exact discounts depend on the GCP service used in the research project and can vary between 5% and 25%. Free data egress is usually included.
Expand All @@ -36,12 +36,13 @@ The NYU Research Cloud team does not currently offer training on how to deploy a
- Through the [Google For Education][google-for-edu] program, GCP offers training credits and discounts to Students, Faculty, and IT Admins. To apply for training credits and discounts, please [click here](https://services.google.com/fb/forms/googlecloudskillsbooststudenttrainingcreditsapplication/).
- [Getting started with Google Cloud Platform][gcp-get-started] offers quick starts and sample projects on GCP.

## How can I fund my research project on GCP?
## How Can I Fund My Research Project on GCP?
### GCP Free Tier
Apply for credits using your NYU account (https://cloud.google.com/free/). After credits expire, if you would like to switch to another type of funding and are approved to do so, we will modify your project so it can use other funds:
- https://edu.google.com/programs/credits/research/?modal_active=none
- https://edu.google.com/programs/?modal_active=none
### Sources of funding for GCP project

### Sources of Funding for GCP Project
Please consider options below and explore other options which may exist for your specific field.
- [Google Cloud research credits][gcp-credits]
- NIH STRIDES
Expand Down
2 changes: 1 addition & 1 deletion docs/cloud/02_gcp_self_managed/02_nih_strides.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ The benefits of participating in NIH Strides program include:
- Receive opportunities for professional service engagements to help drive success
- Receive guidance for best practices in areas such as data storage, governance, and controlled access

## Enrolling to the NIH Strides initiative
## Enrolling to the NIH Strides Initiative

NYU has enrolled in the NIH Strides initiative in December 2020 by signing an agreement with Carahsoft, GCP's billing and administrative partner. Thus NIH-funded NYU researchers with an active NIH award may take advantage of the STRIDES Initiative for their NIH-funded research projects. The NYU RTS team works closely with Burwood Group, a GCP reseller, to provide access to GCP resources for NYU researchers who are approved to participate in the NIH STRIDES initiative. NYU researchers who wish to participate must follow the steps outlined below.

Expand Down
2 changes: 1 addition & 1 deletion docs/cloud/04_dataproc/01_intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

Dataproc is a cloud-based Hadoop distribution that is managed by Google. Google administers updates to Dataproc so that it is kept current. Google also packages and maintains additional software that can be run on top of Hadoop. Additionally Dataproc includes other cloud-specific features, such as the ability to automatically add/remove nodes depending upon how busy the cluster is (autoscaling). It can also use object storage ([GCS][gcs]) or [BigQuery][bigquery] as an alternative to HDFS, and provides integration with [BigTable][bigtable] using HBase interfaces.

### What is Hadoop?
### What Is Hadoop?

Hadoop is an open-source software framework for storing and processing big data in a distributed/parallel fashion on large clusters of commodity hardware. At its core, Hadoop strives to increase processing speed by increasing [data locality][data-locality] (i.e., it moves computation to servers where the data is located). There are three components to Hadoop: HDFS (the Hadoop Distributed File System), the Hadoop implementation of MapReduce, and YARN (Yet Another Resource Negotiator; a scheduler).

Expand Down
2 changes: 1 addition & 1 deletion docs/cloud/04_dataproc/02_data_management.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Data management
# Data Management

HDFS stands for Hadoop Distributed File System. HDFS is a highly fault-tolerant file system and is designed to be deployed on low-cost hardware. HDFS provides high throughput access to application data and is suitable for analyses that use large datasets.

Expand Down
4 changes: 2 additions & 2 deletions docs/genai/02_onboarding/01_intro.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@
This section deals with the eligibility for getting access to Pythia. To learn more about affiliate access (students after graduation, collaborators from other institutions, please refer to [this section](../../hpc/01_getting_started/02_HPC_Accounts/01_getting_and_renewing_an_account.mdx))

:::tip[workspaces]
Access to Pythia is facilitated via workspaces. Beyond facilitating access, they also allow team members to collaboratively dvelop prompts.
Access to Pythia is facilitated via workspaces. Beyond facilitating access, they also allow team members to collaboratively develop prompts.
:::


## Who is eligible for to access Pythia?
## Who Is Eligible for to Access Pythia?
All full-time NYU faculty have the ability to request the creation of workspaces by filling out [the research intake form][research-workspace-request]. Once we create the workspace, members need to submit the [member on-boarding form][member-onboarding-form] to gain access to the workspace. Please be sure to indicate the models you'd like access to from the [catalouge](../03_external_llms/01_catalogue.md)


Expand Down
2 changes: 1 addition & 1 deletion docs/genai/02_onboarding/02_setup.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Setup

## Accessing your workspace
## Accessing Your Workspace
Login to Portkey at [`app.portkey.ai`](https://app.portkey.ai/login) with the `Single-sign-on` option using your NYU NetID. Once you're in, you'll be placed in the "Shared Workspace" by default. You can navigate to your workspace by selecting it from the workspace drop-down list from the top of the left sidebar.

## API Keys
Expand Down
2 changes: 1 addition & 1 deletion docs/genai/02_onboarding/03_quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ If you are sending requests from a server like Colab notebook while your laptop
:::


## Getting started with the LLM gateway
## Getting Started with the LLM Gateway

:::tip[Gateway URL]
Whenever you instantiate a Portkey client, the `base_url` must be set to `base_url="https://ai-gateway.apps.cloud.rt.nyu.edu/v1/"`. If you miss this parameter you would be connecting to the vendor's SaaS platform and the API keys you created after signing on with SSO will not work.
Expand Down
2 changes: 1 addition & 1 deletion docs/genai/04_how_to_guides/01_temperature.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ completion = portkey.chat.completions.create(
print(completion)
```

At the temperature of 2.0, you might get an output along along the lines of:
At the temperature of 2.0, you might get an output along the lines of:

- ``"listening to old radio static."``
- ``"... a really peculiar shade of chartreuse today."``
Expand Down
6 changes: 3 additions & 3 deletions docs/genai/04_how_to_guides/02_embeddings.mdx
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Generating embeddings
# Generating Embeddings

While Decoder-only LLMs gained massive popularity via their usage in chatbots, Encoder-only LLMs can be used for a wider variety of tasks. Decoder-only LLMs "generate" tokens ("text") one at a time probabalisticsally. Encoder-only LLMs on the other hand take text as their input, tokenize it and generate "embeddings" as their output. Here, we shall walk through a task of generating embeddings from a text snippet.
While Decoder-only LLMs gained massive popularity via their usage in chatbots, Encoder-only LLMs can be used for a wider variety of tasks. Decoder-only LLMs "generate" tokens ("text") one at a time probabilistically. Encoder-only LLMs on the other hand take text as their input, tokenize it and generate "embeddings" as their output. Here, we shall walk through a task of generating embeddings from a text snippet.

```mermaid
flowchart LR;
Expand Down Expand Up @@ -40,7 +40,7 @@ and gives the following response:
[0.052587852, 0.094195396, 0.24439038, 0.104940414, -0.028921358, -0.31591928, -0.1846261, 0.221018, 0.033215445, -0.1382735, -0.14776362, -0.15058714, 0.057725072, -0.23435123, 0.07956805, -0.32156628, -0.08454841, 0.04066637, -0.022215525, 0.19090058, -0.11160703, 0.22258662, -0.06843088, -0.22854735, 0.1033718, -0.38085997, 0.2933312, -0.023215517, 0.20768477, -0.039333045, 0.17192031, -0.14180289]
```

## Applications of embeddings
## Applications of Embeddings

Embeddings are typically used for:
- retrieval-augmented generation
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Retrieval-augmented generation
# Retrieval-augmented Generation

:::tip[Foundations of Research Computing session on RAG]
For an in-depth overview of RAG and Jupyter notebook examples, please access the source materials used for the 2025 FORC session on RAG at: https://github.com/NYU-RTS/rag-forc-2025
Expand Down
10 changes: 5 additions & 5 deletions docs/genai/04_how_to_guides/04_batch_inference.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,14 @@ When immediate results are not needed, for instance in transforming large datase
Batch processing is only supported for LLMs that can be accessed via the `@vertexai` provider.
:::

## Collect the prompts
## Collect the Prompts
You'll collect the prompts you want to send to the LLM as a newline delimited JSON ([JSONLines](https://jsonlines.org/)) where each line contains a single prompt in the OpenAI format. Here's an example we will be using in this example:
```json
{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gemini-3-flash-preview", "messages": [{"role": "user", "content": "Where is NYU located?"}], "max_tokens": 2048}}
{"custom_id": "request-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gemini-3-flash-preview", "messages": [{"role": "user", "content": "What resources are available for genAI research at nyu?"}], "max_tokens": 8196}}
```

## Upload them to the GCS bucket
## Upload Them to the GCS Bucket
We will upload this via the Portkey client to the GCS ([Google Cloud Storage](https://cloud.google.com/storage)) bucket via the following script:
```python
from portkey_ai import Portkey
Expand Down Expand Up @@ -50,7 +50,7 @@ This script will print to standard output the location of the uploaded file, lik

```

## Submit a batch inference job
## Submit a Batch Inference Job

We are now ready to submit the batch inference job. Here's a script to do so:
```python
Expand Down Expand Up @@ -78,7 +78,7 @@ print(batch_job)

Upon successful submission, you'll see an `id` field that refers to the job id.

## Query job status
## Query Job Status

Using the id of the batch inference job, you can query the status by:

Expand Down Expand Up @@ -111,7 +111,7 @@ The output for a pending job looks like:
Once the job completes, the `status` field will change from `in_progress` to `completed`.


## Retrieving the output
## Retrieving the Output

The output of the batch inference job can be obtained by:
```sh
Expand Down
2 changes: 1 addition & 1 deletion docs/genai/04_how_to_guides/05_llm_fine_tuning.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Fine tuning
# Fine Tuning

Please look into harnessing RAG before attempting to fine-tune a model. For open-weight models, you can use the HPC cluster to perform LoRA fine-tuning as [described here](../../hpc/08_ml_ai_hpc/05_llm_fine_tuning.md).
2 changes: 1 addition & 1 deletion docs/hpc/01_getting_started/01_intro.mdx
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Start here!
# Start Here!

Welcome to the Torch HPC documentation! If you do not have an HPC account, please proceed to the next section that explains how you may be able to get one.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ Linux clients are not officially supported, however we were able to successfully
sudo openconnect -b vpn.nyu.edu
```
- When prompted follow the instructions and provide your netID, password, and authenticate with ('push', 'phone1' or 'sms')
This method was tested on few Linux distributions and settings however is not guaranteeed to work in future.
This method was tested on few Linux distributions and settings however is not guaranteed to work in future.
</TabItem>
</Tabs>

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# How to request an HPC account
# How to Request an HPC Account

:::tip
Make sure you don't already have an HPC account. You can check this by attempting to log in to the cluster, according to the instructions at [Connecting to the HPC Cluster](../../02_connecting_to_hpc/01_connecting_to_hpc.mdx).
Expand Down Expand Up @@ -36,7 +36,7 @@ You need to be on the NYU VPN to perform this task!
If the **Request HPC Account** QuickLink is clicked, the following form appears:
![Faculty request form](../static/faculty_request_form.png)

The user’s name will be pre populated, and the forms required fields must be completed (sponsor, reason for request, consent to terms of use). After clicking “Submit” the chosen sponsor will be notified of the request and provisioning will only occur after approval.
The user’s name will be prepopulated, and the forms required fields must be completed (sponsor, reason for request, consent to terms of use). After clicking “Submit” the chosen sponsor will be notified of the request and provisioning will only occur after approval.

If the **Bulk HPC Account Request** QuickLink is clicked, the following form appears:
![Bulk account request form](../static/bulk_acct_req_form.png)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Renewing your HPC Account
# Renewing Your HPC Account

:::info VPN Needed
You need to be on the NYU VPN to perform this task!
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# Slurm Accounts

Users are required to have at least one active Slurm acount to submit jobs on the cluster. You will specify this with the `--account=` flag. Accounts are associated with an active allocation within a project that PIs manage using the [HPC Project Portal](02_hpc_project_management_portal.mdx).
Users are required to have at least one active Slurm account to submit jobs on the cluster. You will specify this with the `--account=` flag. Accounts are associated with an active allocation within a project that PIs manage using the [HPC Project Portal](02_hpc_project_management_portal.mdx).
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Managing allocations for your project
# Managing Allocations for Your Project

:::info[VPN Needed]
You need to be connected to [NYU VPN](https://www.nyu.edu/life/information-technology/infrastructure/network-services/vpn.html) to access the HPC project management portal.
Expand All @@ -13,10 +13,10 @@ Login to HPC projects portal at [projects.hpc.nyu.edu](https://projects.hpc.nyu.
From the list of project, click on the project you'd like to submit an allocation request for and you'll see the current details:
!["PI_One Project" section](../static/PI_one_project.png)

## Current allocations
## Current Allocations
Scroll down and you'll reach the Allocations section. This section lists all the allocations associated with this project. All allocations have SLURM accounts associated with them. Allocations that are "Active" will allow you to submit jobs using the SLURM account associated with it.

## Requesting new allocations
## Requesting New Allocations
If you scroll down, you'll see "+Request Resource Allocation" button.
!["PI request allocation" section](../static/PI_request_allocation.png)

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Entering Grants and Publciations for Your Project
# Entering Grants and Publications for Your Project

## Acknowledgment Statement for Publications
The following acknowledgment statement should appear in the publication of any material that resulted from using the NYU IT HPC resources, services, and staff expertise.
Expand Down
4 changes: 2 additions & 2 deletions docs/hpc/02_connecting_to_hpc/01_connecting_to_hpc.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Host torch login.torch.hpc.nyu.edu
```

:::warning[No SSH Keys on Torch]
SSH keys are not supported on Torch due to increased security resrtictions.
SSH keys are not supported on Torch due to increased security restrictions.
:::


Expand Down Expand Up @@ -99,7 +99,7 @@ You can access the ssh client via Powershell, either by installing [Windows Term

Alternatively, you can install WSL2, and then install Ubuntu or other Linux distribution (for example, from Microsoft Store). You will have a fully functional Ubuntu with terminal and can connect to cluster using instructions provided above for Linux/Mac users. Instructions on WSL installation can be found here: [https://docs.microsoft.com/en-us/windows/wsl/install-win10][wsl installation link]

With [Windowns Terminal](https://apps.microsoft.com/detail/9n0dx20hk701?hl=en-US&gl=US), you can access both the Linux WSL2 shell and the Windows Powershell from the same application.
With [Windows Terminal](https://apps.microsoft.com/detail/9n0dx20hk701?hl=en-US&gl=US), you can access both the Linux WSL2 shell and the Windows Powershell from the same application.

:::tip
- If you are using WSL 2 (Windows subsystem for Linux 2), you may not be able to access internet when Cisco AnyConnect VPN, installed from exe file, is activated. A potential solution: uninstall Cisco AnyConnect and install AnyConnect using Microsoft Store, and then setup new VPN connection using settings described on [IT webpage][install vpn on windows link].
Expand Down
6 changes: 3 additions & 3 deletions docs/hpc/03_storage/02_data_transfers.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
Globus is the recommended tool to use for large-volume data transfers due to the efficiency, reliability, security and ease of use. Use other tools only if you really need to. Detailed instructions available at [Globus](./03_globus.md)
:::

## Data-Transfer nodes
## Data-Transfer Nodes
Attached to the NYU HPC cluster Torch, the Torch Data Transfer Node (DTN) are nodes optimized for transferring data between cluster file systems (e.g. scratch) and other endpoints outside the NYU HPC clusters, including user laptops and desktops. The gDTNs have 100-Gb/s Ethernet connections to the High Speed Research Network (HSRN) and are connected to the HDR Infiniband fabric of the HPC clusters. More information on the hardware characteristics is available at [Torch spec sheet](../10_spec_sheet.md).

### Data Transfer Node Access
Expand All @@ -26,7 +26,7 @@ logout

### Linux & Mac Tools

#### `scp` And `rsync`
#### `scp` and `rsync`
:::warning
Please use Data Transfer Nodes (DTNs) with these tools. While one can transfer data while on login nodes, it is considered a bad practice because it can degrade the node's performance.
:::
Expand All @@ -50,7 +50,7 @@ Globus is the recommended tool to use for large-volume data transfers. It featur

The Globus endpoint for Torch is available at `nyu#torch`. Detailed instructions available at [Globus](./03_globus.md)

### rclone
### `rclone`
rclone - rsync for cloud storage, is a command line program to sync files and directories to and from cloud storage systems such as Google Drive, Amazon Drive, S3, B2 etc. rclone is available on DTNs. [Please see the documentation for how to use it.](https://rclone.org/)

### Open OnDemand (OOD)
Expand Down
Loading
Loading