When you load tables to CAS or run CAS actions, the following error might occur:

The error message refers to the CAS disk cache location, which is also known as CAS_DISK_CACHE.

The most common reasons for this error are as follows:

• The CAS_DISK_CACHE location is not writable by the CAS server process owner or the CAS session user on one or more CAS machines.
• There is not enough free space in CAS_DISK_CACHE for the table that is being loaded or for temporary files generated by CAS actions.
• A CAS process has reached the maximum open file descriptor limit (ulimit -n) or maximum file size limit (ulimit -f) that is effective for the user that owns the process.

The sections below explain how to handle each of these scenarios.

 

Find Your CAS_DISK_CACHE Location

There are multiple ways to find your CAS_DISK_CACHE location.

1. In SAS^® Environment Manager, navigate to Servers ► cas-shared-default (or cas-tenant-default for tenant CAS servers) ► Configuration ► Choose a CAS node ► Runtime Environment. 

2. In the Runtime Environment list, find the CAS_DISK_CACHE environment variable.

3. Note the value of the CAS_DISK_CACHE variable, which is the disk cache location. This path should exist on all CAS controllers and workers, unless the CAS_CONTROLLER_TEMP variable is set so that an alternative location is used on the CAS controller machines only.

 

Determine the Space Needed in CAS_DISK_CACHE

You need enough space in CAS_DISK_CACHE for all tables that are loaded to CAS, plus any table copies that are created for redundancy in distributed CAS environments. Note that most load operations in distributed CAS environments create one table copy by default. For example, a rough estimate is that a 50 GB table loaded to a distributed CAS server will require 100 GB in CAS_DISK_CACHE. This space accounts for an additional copy of the table due to the default replication factor of 1. CAS programmers can disable copies if desired via the copies=0 (PROC CASUTIL and DATA step) or replication=0 (loadTable action) options.

The only tables that do not require writing to CAS_DISK_CACHE are tables loaded from SASHDAT files that are available to CAS for direct memory mapping. This includes SASHDAT files accessed via DNFS or co-located HDFS caslibs in distributed CAS environments and SASHDAT files accessed via PATH caslibs in single-machine CAS environments.

 

Change Your CAS_DISK_CACHE Location

If you need to change your CAS_DISK_CACHE to a location with more space, set the new value in the /opt/sas/viya/config/etc/cas/default/casconfig_usermods.lua file on the CAS controller and restart sas-viya-cascontroller-default to make the change effective. Here is an example:

The path(s) provided in this variable must exist on all CAS nodes and must be writable by all users who will run CAS session processes.  

 

Which users need Write access to CAS_DISK_CACHE and high ulimit settings?

The user who runs the CAS server and any users that run CAS session processes under their own host accounts need Write access to CAS_DISK_CACHE and high ulimit settings for maximum open file descriptors and maximum file size. 

The default CAS server process owner is cas, but you can customize this owner during deployment. You can find the server process owner in the first line of the SAS^® Viya^® 3.5 CAS logs, as shown below:

In releases earlier than SAS Viya 3.5, the server process owner is the user name found on log lines containing "New client connection accepted" messages, as in this example:

Users who connect to CAS from programming environments (SAS^® 9.4 clients, SAS^® Studio Basic, and so on) and users in the CASHostAccountRequired custom group also need Write access to CAS_DISK_CACHE and high ulimits.

 

What are the recommended ulimit values?

Since CAS creates many small files/blocks in CAS_DISK_CACHE and maintains open file descriptors to these blocks until the table is dropped or CAS is stopped, the maximum open file descriptors limit (nofile) should be high, such as 100,000 or greater. The maximum file size limit (fsize) should be unlimited, which is often the OS default.

 

Why can I not see any files in the CAS_DISK_CACHE location? Why are there so many deleted files still in use by CAS processes?

CAS uses "delete on last close" behavior for files in CAS_DISK_CACHE. The files are not visible except via tools like lsof or by listing the /proc/<pid>/fd symlinks to deleted files. The files will be fully deleted and space in CAS_DISK_CACHE freed when any of the following occur:

• The table that these files represent is unloaded/dropped from CAS, either manually or because they are session tables and the CAS session terminates.
• The CAS server is stopped.