Skip to content
Data Engine Thinking Data Engine Thinking Samples

Domain-Specific Languages

A domain-specific language (DSL) provides an alternative to a general-purpose templating engine. This can be demonstrated with a sample template script that uses the Biml DSL. Consider the code below, a mixture of C# and the template using Biml (XML) syntax.

<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<#
var mappingList = new List<DataObjectMappings>();
var allFiles = Directory.GetFiles(@"<directory containing JSON files>");
foreach (var fileName in allFiles)
{
   // Fetch the content of the Json files
   string jsonInput = File.ReadAllText(fileName);
   var deserialisedMappings = JsonConvert.DeserializeObject<DataObjectMappings>(jsonInput);
   mappingList.Add(deserialisedMappings);
}
#>
   <Packages>
      <#
      foreach (var dataObjectMappings in mappingList)
      {
         int counter = 1;
         foreach (var dataObjectMapping in dataObjectMappings.dataObjectMappings.ToList())
         {
         #>
   <Package Name="<#=dataObjectMapping.name#><#=counter#>">
     <Tasks>
       <Dataflow Name="DF - <#=dataObjectMapping.name#><#=counter#> - 1">
       </Dataflow>
       <Dataflow Name="DF - <#=dataObjectMapping.name#><#=counter#> - 2">
         <PrecedenceConstraints>
           <Inputs>
             <Input OutputPathName="DF - <#=dataObjectMapping.name#><#=counter#> - 1.Output" />
           </Inputs>
         </PrecedenceConstraints>
       </Dataflow>
     </Tasks>
   </Package>
         <#
         counter++;
         }
      }
      #>
   </Packages>
</Biml>

The C# blocks (<# … #>) read every JSON metadata file in a directory and deserialize each one into the same data object mappings used throughout these samples. For each mapping, the script emits an SSIS package with two chained data flow tasks — the second running after the first through a precedence constraint.

Executing the script

Section titled “Executing the script”

Biml is processed by a Biml compiler, which runs the embedded C# to expand the metadata and then generates the target packages. The most common free option is BimlExpress, an add-in for Visual Studio; Biml Studio is the commercial IDE.

A typical run with BimlExpress:

  1. Replace <directory containing JSON files> with the folder that holds your JSON mapping files.
  2. Make the DataObjectMappings object model and Json.NET available to the script (for example, by referencing the schema’s class library).
  3. Add the .biml file to an Integration Services project in Visual Studio, right-click it, and choose Generate SSIS Packages.

The compiler expands the BimlScript — reading and deserializing each metadata file — and emits the packages.

Expected output

Section titled “Expected output”

The expansion produces one package per data object mapping. For a mapping named Customer, the script expands to:

<Packages>
  <Package Name="Customer1">
    <Tasks>
      <Dataflow Name="DF - Customer1 - 1">
      </Dataflow>
      <Dataflow Name="DF - Customer1 - 2">
        <PrecedenceConstraints>
          <Inputs>
            <Input OutputPathName="DF - Customer1 - 1.Output" />
          </Inputs>
        </PrecedenceConstraints>
      </Dataflow>
    </Tasks>
  </Package>
</Packages>

Compiling this Biml produces the corresponding SSIS packages (.dtsx), each containing the two data flow tasks shown. Biml can also target Azure — generating Azure Data Factory pipelines and mapping data flows — from the same metadata-driven approach.