The %MDUEXTR macro fails with a data conversion error


When running the %MDUEXTR macro alone or as part of the user bulk load or user synchronization sample programs, the following error might occur:

ERROR: IOM call failed because of a data conversion error.

ERROR: Failed to transcode data from U_UTF8_CE to U_WLATIN1_CE encoding because it contained characters which are not supported by your SAS session encoding.

This issue often occurs after the macro runs successfully for some time.

The root cause of this behavior is that SAS® Metadata Server runs in UTF-8/Unicode session encoding, which allows non-ASCII characters to be stored in the metadata. If you run a SAS session that does not use UTF-8/Unicode encoding and the metadata query returns non-ASCII characters, the SAS session is unable to process the characters.

Workarounds:

To circumvent this issue, either run your program in a UTF-8 encoded SAS session or remove the non-ASCII characters from SAS Metadata Server.

Run your program in a UTF-8 encoded SAS Session

To run your program in a UTF-8 encoded SAS Session, see Create a Unicode Application Server Context. This documentation will help you add a Unicode application server context (for example, SASAppU8) to your SAS 9.4 installation. 

Remove non-ASCII Characters from Metadata

To identify the non-ASCII characters that are being returned in order to remove them, complete the following steps.

UNIX

Note: This workaround requires that the xmllint command is available to format the XML output.

  1. Run a UTF-8 SAS Session on the SAS Metadata Server host by running the following as the SAS installation account:

<SASHome>/SASFoundation/9.4/bin/sas_u8 -nodms 

  1. Run the code below to capture the XML gathered by the MDUEXTR macro into a file called /tmp/issue.xml:

%include "/opt/sas/M8LSF/config/Lev1/SASMeta/MetadataServer/metaparms.sas";
%mdumap(mapname=USERINFO, mapref=_USERMAP, queryref=_USERQRY );
filename _metpinf "/tmp/issue.xml";
PROC METADATA
   header=full
   out=_metpinf
   in=_USERQRY;
  RUN;
endsas;

  1. Run the command below to write the lines containing non-ASCII characters to the terminal:

xmllint --format /tmp/issue.xml | grep -P '[^[:ascii:]]'

Windows

  1. Run a UTF-8 SAS Session by logging on to the Metadata Server host as the SAS installation account. Then, click StartSASSAS 9.4 (Unicode Support).
  2. Run the code below to capture the XML gathered by the MDUEXTR macro into a file called C:\Temp\issue.xml:

%include "<SASConfig>\Lev1\SASMeta\MetadataServer\metaparms.sas";
%mdumap(mapname=USERINFO, mapref=_USERMAP, queryref=_USERQRY );
filename outxml "C:\Temp\issue.xml";
PROC METADATA
   header=full
   out=outxml
   in=_USERQRY;
  RUN;

  1. Run these PowerShell commands to format the generated XML file into a new file called C:\Temp\issue_formatted.xml and write any lines containing non-ASCII characters to the terminal:

PS C:\> $xml = [xml] (Get-Content "C:\Temp\issue.xml")
PS C:\> $xml.Save("C:\Temp\issue_formatted.xml")
PS C:\> Select-String -allmatches -pattern "[^\x00-\x7F]" -LiteralPath "C:\Temp\issue_formatted.xml"