{"id":390,"date":"2020-10-02T12:35:42","date_gmt":"2020-10-02T17:35:42","guid":{"rendered":"https:\/\/computing.fnal.gov\/wilsoncluster\/?page_id=390"},"modified":"2024-01-19T15:42:33","modified_gmt":"2024-01-19T21:42:33","slug":"filesystems","status":"publish","type":"page","link":"https:\/\/computing.fnal.gov\/wilsoncluster\/filesystems\/","title":{"rendered":"Filesystems"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"545\" src=\"https:\/\/computing.fnal.gov\/wilsoncluster\/wp-content\/uploads\/2020\/12\/Screen-Shot-2020-12-17-at-2.48.32-PM-1-1024x545.png\" alt=\"\" class=\"wp-image-2030\" srcset=\"https:\/\/computing.fnal.gov\/wilsoncluster\/wp-content\/uploads\/2020\/12\/Screen-Shot-2020-12-17-at-2.48.32-PM-1-1024x545.png 1024w, https:\/\/computing.fnal.gov\/wilsoncluster\/wp-content\/uploads\/2020\/12\/Screen-Shot-2020-12-17-at-2.48.32-PM-1-300x160.png 300w, https:\/\/computing.fnal.gov\/wilsoncluster\/wp-content\/uploads\/2020\/12\/Screen-Shot-2020-12-17-at-2.48.32-PM-1-768x409.png 768w, https:\/\/computing.fnal.gov\/wilsoncluster\/wp-content\/uploads\/2020\/12\/Screen-Shot-2020-12-17-at-2.48.32-PM-1-1536x818.png 1536w, https:\/\/computing.fnal.gov\/wilsoncluster\/wp-content\/uploads\/2020\/12\/Screen-Shot-2020-12-17-at-2.48.32-PM-1-2048x1091.png 2048w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-vivid-red-color\">Important considerations about home directories in \/nashome<\/mark><\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong><mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-vivid-red-color\">You must have ssh delegate (forward) your Kerberos ticket when you login to the cluster, otherwise, your home directory in the \/nashome filesystem will not be accessible.<\/mark><\/strong><\/li>\n\n\n\n<li><strong><mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-vivid-red-color\">Do not launch batch jobs from \/nashome. They will silently fail without a trace due to not having a valid Kerberos ticket!<\/mark><\/strong><\/li>\n\n\n\n<li><strong><mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-vivid-red-color\">Batch jobs should not depend on access to \/nashome in any way including software intended to be used in batch jobs.<\/mark><\/strong> <strong><mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-vivid-red-color\">Many software packages including Python, conda, pip, Julia, R, Spack, EasyBuild, and Apptainer use (hidden) directories in $HOME by default. Users should make use of relevant shell environment variables specific to such packages to place these directories in their work directory in the \/work1 filesystem.<\/mark><\/strong><\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">filesystems available on the Wilson Cluster<\/h2>\n\n\n\n<p>The various filesystems are listed in the table below<\/p>\n\n\n\n<figure class=\"wp-block-table has-small-font-size\"><table><tbody><tr><td><strong>Area<\/strong><\/td><td><strong>Description<\/strong><\/td><\/tr><tr><td><mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-black-color\">\/nashome<\/mark><\/td><td><strong>User home area<\/strong>.<br>Quota limit per user = 5 GB.&nbsp;<br>By default, every user with a valid account on WC-IC has a home directory under this area.<br><strong>Backup: YES<\/strong><\/td><\/tr><tr><td>\/work1<\/td><td><strong>Project data area<\/strong>. Permissions and quota managed by GID\/group.  <br>Default quota limit per project = 25GB. <br>Projects may request additional space using the Project and User Requests forms.<br><strong>Backup: NO<\/strong><\/td><\/tr><tr><td>\/cvmfs<\/td><td><strong>CernVM File System<\/strong>. <br>Access to HEP software from WC-IC login hosts and worker nodes.<\/td><\/tr><tr><td>\/wclustre<\/td><td><strong>Lustre storage<\/strong>.&nbsp;<br>Default quota limit per group = 100GB.<br>Visible on all cluster worker nodes.<br>Ideal for temporary storage (~month) of very large data files. Because Lustre is optimized for large files, groups of small files should be packed into larger tar files. <br>Disk space usage is monitored, and disk quotas enforced.&nbsp; <br>One of the main factors leading to the high performance of Lustre\u2122 file systems is the ability to stripe data over multiple OSTs. The stripe count can be set on a file system, directory, or file level. See the <a href=\"https:\/\/wiki.lustre.org\/Configuring_Lustre_File_Striping\" target=\"_blank\" rel=\"noreferrer noopener\">Lustre Striping Guide<\/a> for more information.<br><strong>Backup: NO<\/strong><\/td><\/tr><tr><td>\/scratch<\/td><td><strong>Local scratch disk<\/strong> on each worker node.&nbsp;<br>289GB in size. <br>Suitable for writing all sorts of data created in a batch job. Offers the best performance.<br>Any precious data needs to be copied off of this area before a job ends since the area is automatically purged at the end of a job.<br><strong>Backup: NO<\/strong><\/td><\/tr><tr><td>\/pnfs\/fermigrid<\/td><td>Fermigrid <span class=\"has-inline-color has-vivid-red-color\">volatile<\/span> <strong>dCache<\/strong> storage.<br>Visible on cluster submit host only.<br>Ideal for storage of parameter files and results.<br><strong>Backup: NO<\/strong><\/td><\/tr><tr><td>Globus<\/td><td><strong>Globus End-Point <\/strong>named <span class=\"has-inline-color has-vivid-red-color\">fnal#wilson<\/span> which you can use to transfer data in and out of WC-IC. Please refer to the documentation on <a rel=\"noreferrer noopener\" href=\"https:\/\/www.globus.org\/globus-connect-personal\" target=\"_blank\">globus.org<\/a> on setting up a Globus Connect Personal if needed.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Accessing internet addresses from worker nodes<\/h2>\n\n\n\n<p>A web proxy is needed to access internet url&#8217;s from the worker nodes. Use cases include downloading software via <code>git<\/code> or using <code>wget<\/code> to fetch files. Set the environment variables below from your shell running on a worker node.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>export https_proxy=http:\/\/squid.fnal.gov:3128\nexport http_proxy=http:\/\/squid.fnal.gov:3128<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Other Documentation<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>User documentation for Fermilab tape access tools is found in the <a rel=\"noreferrer noopener\" href=\"http:\/\/cd-docdb.fnal.gov\/cgi-bin\/RetrieveFile?docid=5399&amp;filename=PublicdCacheHowTo.pdf&amp;version=4\" target=\"_blank\">Enstore\/dCache User&#8217;s Guide<\/a>.<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Important considerations about home directories in \/nashome filesystems available on the Wilson Cluster The various filesystems are listed in the table below Area Description \/nashome User home area.Quota limit per user = 5 GB.&nbsp;By default, every user with a valid account on WC-IC has a home directory under this area.Backup: YES \/work1 Project data area&#8230;. <a class=\"more-link\" href=\"https:\/\/computing.fnal.gov\/wilsoncluster\/filesystems\/\"> More &#187;<\/a><\/p>\n","protected":false},"author":15,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":"[]"},"class_list":["post-390","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/computing.fnal.gov\/wilsoncluster\/wp-json\/wp\/v2\/pages\/390","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/computing.fnal.gov\/wilsoncluster\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/computing.fnal.gov\/wilsoncluster\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/computing.fnal.gov\/wilsoncluster\/wp-json\/wp\/v2\/users\/15"}],"replies":[{"embeddable":true,"href":"https:\/\/computing.fnal.gov\/wilsoncluster\/wp-json\/wp\/v2\/comments?post=390"}],"version-history":[{"count":40,"href":"https:\/\/computing.fnal.gov\/wilsoncluster\/wp-json\/wp\/v2\/pages\/390\/revisions"}],"predecessor-version":[{"id":7079,"href":"https:\/\/computing.fnal.gov\/wilsoncluster\/wp-json\/wp\/v2\/pages\/390\/revisions\/7079"}],"wp:attachment":[{"href":"https:\/\/computing.fnal.gov\/wilsoncluster\/wp-json\/wp\/v2\/media?parent=390"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}