The error "Some character data was lost during transcoding in the data set" occurs when the data set encoding does not match the SAS® session encoding


The following note and error message indicate that a SAS® data set has an encoding value that does not match the encoding setting for the SAS session:

NOTE: Data file libref.datasetname.DATA is in a format that is native to another host,
      or the file encoding does not match the session encoding. Cross Environment Data
      Access will be used, which might require additional CPU resources and might reduce
      performance.
ERROR: Some character data was lost during transcoding in the data set
       libref.datasetname. Either the data contains characters that are not
       representable in the new encoding or truncation occurred during transcoding.

In order to confirm the mismatch, use the following code for further debugging where libref.datasetname is the name of the problematic data set:

/* Show the configuration file and the encoding setting for the SAS session */
proc options option=config; run;
proc options group=languagecontrol; run; 
/* Show the encoding value for the problematic data set */
%let dsn=libref.datasetname;
%let dsid=%sysfunc(open(&dsn,i));
%put &dsn ENCODING is: %sysfunc(attrc(&dsid,encoding));
%let rc=%sysfunc(close(&dsid));

Compare the encoding value setting of the SAS session with the encoding value setting of the data set. If the two do not match, then from SAS 9.4 you can either change the SAS session encoding to match that of the data set, or change the data set encoding to match that of the SAS session encoding. If you are running SAS® Viya® where the default encoding is UTF-8, then a conversion of the encoding value for the SAS data set is necessary. Sample code is shown in SAS KB0039558, "How to convert SAS® data set encoding."

If you are running SAS 9.4, the CONFIG information from the OPTIONS procedure displays in the SAS log the config file that is currently in use, usually !SASHome\SASFoundation\9.x\nls. The same directory contains other configuration files, at a minimum: en (English), 1d (DBCS), and u8 (Unicode). If you choose to change the encoding of the SAS session to match the encoding of the SAS data set, locate the desired configuration file, and use this file to invoke SAS.

In the following use case, the data set shows UTF-8 encoding, and the SAS® 9.4 session shows Wlatin1 encoding:

Log window shows:

libref.datasetname ENCODING is: utf-8 Unicode (UTF-8)

OPTIONS procedure output:

CONFIG=C:\Program Files\SASHome94\SASFoundation\9.4\nls\en\sasv9.cfg
ENCODING=WLATIN1

Your SAS session should be invoked with UTF-8 encoding in order to avoid the NOTE and ERROR.

To invoke SAS 9.4 using UTF-8 encoding on Windows, do one of the following:

 

1. Right-click the SAS icon and select Properties. On the Shortcut tab, in the Target line, add the desired configuration file (with the complete path) to the end:

-config "C:\Program Files\SASHome94\SASFoundation\9.4\nls\u8\sasv9.cfg"

2. Click OK.

On UNIX platforms, the SAS invocation scripts are located in:

 !SASROOT\bin\  

For example:

More information specific to UNIX platforms can be found under Post-installation Configuration for National Language Support.

The following link contains additional NLS documentation: SAS® National Language Support (NLS).

Starting in SAS® 9.4M5 (TS1M5), you can use the %VALIDCHS macro function to validate the character encoding compatibility for data set variables.