web/manual/schema.xml

<?xml version="1.0" encoding="ISO-8859-1" standalone="yes"?>
<!DOCTYPE page [
  <!ENTITY larr     "&#8592;"> <!-- leftwards arrow, U+2190 ISOnum -->
  <!ENTITY uarr     "&#8593;"> <!-- upwards arrow, U+2191 ISOnum-->
  <!ENTITY rarr     "&#8594;"> <!-- rightwards arrow, U+2192 ISOnum -->
  <!ENTITY darr     "&#8595;"> <!-- downwards arrow, U+2193 ISOnum -->
]>
<page name="manual_schema">

<title>XML Schema</title>

<box title="Overview" link="overview">
  <p>
    CDuce partially supports <a href="http://www.w3.org/XML/Schema">XML
      Schema</a> Recommendations (<a
      href="http://www.w3.org/TR/xmlschema-0/">Primer</a>, <a
      href="http://www.w3.org/TR/xmlschema-1/">Structures</a>, <a
      href="http://www.w3.org/TR/xmlschema-2/">Datatypes</a>). Using this CDuce
    feature is possible to manipulate XML documents whose leaves are typed
    values like integers, dates, binary data, and so on.
  </p>
  <p>
    CDuce supports XML Schema by implementing the following features:
  </p>
  <ul>
    <li>
      <a href="#import">XML Schema components import</a>
    </li>
    <li>
      <a href="#validation">XML Schema validation</a>
    </li>
    <li>
      <a href="#print_xml">XML Schema instances output</a>
    </li>
  </ul>
  <p>
    This manual page describes how to use these features in CDuce, all the
    documents used in the examples are available in the manual section: <local
      href="manual_schema_samples">XML Schema sample documents</local>.
  </p>

  <note>
    The support for XML Schema does not currently interact well with
    separate compilation. When a CDuce unit <code>{{script}}.cd</code>
    which uses an XML Schema
    is compiled, the resulting <code>{{script}}.cdo</code> object
    refers to the XML Schema by name. That is, when these units
    are run, the XML Schema must still be available from the current
    directory and must not have been changed since compilation.
  </note>       

</box>

<box title="XML Schema components (micro) introduction" link="primer">
  <p>
    An XML Schema document could define five different kinds of component, each
    of them could be imported in CDuce and used as CDuce types:
  </p>
  <ul>
    <li>
      <b>Type definitions</b><br />
      A type definition defines either a simple type or a complex type. The
      former could be used to type more precisely the string content of an
      element. You can think at it as a refinement of #PCDATA. XML Schema
      provides a set of <a
        href="http://www.w3.org/TR/xmlschema-2/#built-in-datatypes">predefined
        simple types</a> and a way to define new simple types.  The latter could
      be used to constraint the content model and the attributes of an XML
      element. An XML Schema complex type is strictly more expressive than a DTD
      element declaration.
    </li>
    <li>
      <b>Attribute declaration</b><br />
      An attribute declaration links an attribute name to a simple type.
      Optionally it can constraints the set of possible values for the attribute
      mandating a fixed value or providing a default value.
    </li>
    <li>
      <b>Element declarations</b>
      An element declaration links an attribute name to a complex type.
      Optionally, if the type is a simple type, it can constraints the set of
      possible values for the element mandating a fixed value or providing a
      default value.
    </li>
    <li>
      <b>Attribute group definitions</b>
      An attribute group definitions links a set of attribute declarations to a
      name which can be referenced from other XML Schema components.
    </li>
    <li>
      <b>Model group definitions</b>
      A model group definition links a name to a constraint over the complex
      content of an XML element. The linked name can be referenced from other
      XML Schema components.
    </li>
  </ul>
</box>

<box title="XML Schema components import" link="import">
  <p>
    In order to import XML Schema components in CDuce, you first need to tell
    CDuce to import an XML Schema document. You can do this using the
    <code>schema</code> keyword to bind an uppercase identifier to a local
    schema document:
  </p>
  <sample>
# {{schema Mails = "tests/schema/mails.xsd"}};;
Registering schema type: Mails # attachmentType
Registering schema type: Mails # mimeTopLevelType
Registering schema type: Mails # mailType
Registering schema type: Mails # envelopeType
Registering schema type: Mails # mailsType
Registering schema type: Mails # bodyType
Registering schema attribute: Mails # name
Registering schema element: Mails # Date
Registering schema element: Mails # mails
Registering schema element: Mails # header
Registering schema attribute group: Mails # mimeTypeAttributes
Registering schema model group: Mails # attachmentContent
  </sample>
  <p>
    The above declaration will (try to) import all schema components included in
    the schema document <local href="manual_schema_samples">mails.xsd</local>
    as CDuce types. You can reference them using the
    <code>#</code> (sharp) operator.
  </p>
  <p>
    XML Schema permits ambiguity in components name, this implies that you can
    have both an element declaration and an attribute declaration having the
    same name in a single schema document. In case of no ambiguity you can
    reference CDuce types corresponding to schema components just using the name
    with the following syntax:<br /> <tt>schema_ref ::= </tt>
    <code>&lt;schema_name&gt; # &lt;component_name&gt;</code><br />
    Otherwise you can specify the kind of schema component as follows:<br />
    <tt>|</tt> <code>&lt;schema_name&gt; # &lt;component_name&gt; as
      &lt;component_kind&gt;</code><br /> where component kind is one of:<br />
    <tt>component_kind ::= </tt>
    <code>element | type | attribute | attribute_group | model_group</code>
    <br />
  </p>
  <p>
    The result of a schema component reference is an ordinary CDuce type which
    you can use as usual in function definitions, pattern matching and so on.
  </p>
  <sample>
let is_valid_mail (Any -> Bool)
  | {{Mails # mailType}} -> `true
  | _ -> `false
  </sample>
  <p>
    <em>
      Please note the spaces which surround the sharp character, they are
      needed, otherwise <code>#mailType</code> will be considered by the lexer
      as a(n unexistent) directive.
    </em>
  </p>
</box>
<box>
  <p>
    <em>
      <b>Correctness remark:</b> while parsing XML Schema documents, CDuce
      assumes that they're correct with respect to XML Schema recommendations.
      At minimum they're required to be valid with respect to <a
        href="http://www.w3.org/TR/xmlschema-1/#normative-schemaSchema">XML
        Schema for Schemas</a>. It's recommended that you will check for
      validity your schemas before importing them in CDuce, strange behaviour is
      assured otherwise.
    </em>
  </p>
</box>

<box title="Toplevel directives" link="directives">
  <p>
    The toplevel directive <code>#env</code> supports schemas, it lists the
    currently defined schemas.
  </p>
  <sample>
# #env;;
Types: Empty Any Int Char Byte Atom Pair Arrow Record String Latin1 Bool
Namespace prefixes:
=>""
xml=>"http://www.w3.org/XML/1998/namespace"
Namespace prefixes used for pretty-printing:
{{Schemas: Mails}}
Values:
val argv : [ String* ] = ""
  </sample>
  <p>
    The toplevel directive <code>#print_type</code> supports schemas too, it can
    be used to print types corresponding to schema components with the usual
    sharp syntax.
  </p>
  <sample>
# #print_type {{Mails # bodyType}};;
[ Char ]
  </sample>
  <p>
    The toplevel directive <code>#print_schema</code> is not really user
    friendly (because it shows some representation internals), but can be used
    to show the various schema components contained in a given schema.
  </p>
  <sample><![CDATA[
# #print_schema Mails;;
Types: C:10:attachmentType S:mimeTopLevelType' C:12:mailType C:4:envelopeType 
C:14:mailsType S:bodyType' 
Attributes: @name:xsd:string 
Elements: E:18:<Date> E:15:<mails> E:17:<header> 
Attribute groups: {agroup:mimeTypeAttributes} 
Model groups: {mgroup:attachmentContent} 
]]></sample>
  <p>
    For more information have a look at the manual section about <local
      href="manual_interpreter">toplevel directives</local>.
  </p>
</box>

<box title="XML Schema &rarr; CDuce mapping" link="mapping">
  <ul>
    <li>
      <p>
        XML Schema <b>predefined simple types</b> are mapped to CDuce types
        directly in the CDuce implementation preserving as most as possible XML
        Schema constraints. The table below lists the most significant mappings.
      </p>
      <table border="1">
        <tr>
          <td><b>XML Schema predefined simple type</b></td>
          <td><b>CDuce type</b></td>
        </tr>
        <tr>
          <td>
            <code>duration</code>, <code>dateTime</code>, <code>time</code>,
            <code>date</code>, <code>gYear</code>, <code>gMonth</code>, ...
          </td>
          <td>
            closed record types with some of the following fields (depending on
            the Schema type): <code>year</code>, <code>month</code>,
            <code>day</code>, <code>hour</code>, <code>minute</code>,
            <code>second</code>, <code>timezone</code>
          </td>
        </tr>
        <tr><td><code>boolean</code></td><td><code>Bool</code></td></tr>
        <tr>
          <td>
            <code>anySimpleType</code>, <code>string</code>,
            <code>base64Binary</code>, <code>hexBinary</code>,
            <code>anyURI</code>
          </td>
          <td><code>String</code></td>
        </tr>
        <tr><td><code>integer</code></td><td><code>Int</code></td></tr>
        <tr>
          <td>
            <code>nonPositiveInteger</code>, <code>negativeInteger</code>,
            <code>nonNegativeInteger</code>, <code>positiveInteger</code>,
            <code>long</code>, <code>int</code>, <code>short</code>,
            <code>byte</code>
          </td>
          <td>integer intervals with the appropriate limits</td>
        </tr>
        <tr>
          <td> <code>string</code>, <code>normalizedString</code>, and the other
            types derived (directly or indirectly) by restriction from string
          </td>
          <td>String</td>
        </tr>
        <tr>
          <td>
            <code>NMTOKENS</code>, <code>IDREFS</code>, <code>ENTITIES</code>
          </td>
          <td>
            <code>String</code> list (i.e. Kleene star of a <code>String</code>
            type)
          </td>
        </tr>
        <tr>
          <td>
            (<b>Not properly supported</b>)<br /> <code>decimal</code>,
            <code>float</code>, <code>double</code>, <code>NOTATION</code>,
            <code>QName</code>
          </td>
          <td>
            <code>String</code>
          </td>
        </tr>
      </table>
      <p>
        <b>Simple type definitions</b> are built from the above types following
        the XML Schema derivation rules.
      </p>
    </li>
    <li>
      <p>
        XML Schema <b>complex type definitions</b> are mapped to CDuce types
        representing XML elements which can have any tag, but whose attributes
        and content are constrained to be valid with respect to the original
        complex type.
      </p>
      <p>
        As an example, the following XML Schema complex type (a simplified
        version of the homonymous <code>envelopeType</code> defined in <local
          href="manual_schema_samples">mails.xsd</local>):
      </p>
      <sample><![CDATA[
 <xsd:complexType name="envelopeType">
  <xsd:sequence>
   <xsd:element name="From" type="xsd:string"/>
   <xsd:element name="To" type="xsd:string"/>
   <xsd:element name="Date" type="xsd:dateTime"/>
   <xsd:element name="Subject" type="xsd:string"/>
  </xsd:sequence>
 </xsd:complexType>
]]></sample>
      <p>
        will be mapped to an XML CDuce type which must have a <tt>From</tt>
        attribute of type String and four children. Among them the <tt>Date</tt>
        children must be an XML element containing a record which represents a
        <tt>dateTime</tt> Schema type.
      </p>
<sample><![CDATA[
# #print_type Mails # envelopeType;;
<(Any) {| |}>[
  <From {| |}>String
  <To {| |}>String
  <Date {| |}>{
    positive = Bool;
    year = Int; month = Int; day = Int;
    hour = Int; minute = Int; second = Int;
    timezone =? { positive = Bool; hour = Int; minute = Int }
  }
  <Subject {| |}>String
]
]]></sample>
    </li>
    <li>
      <p>
        XML Schema <b>attribute declarations</b> are converted to closed record
        types with exactly one required field corresponding to the declared
        attribute.
      </p>
      <sample>
# #print_type Mails # name;;
{| {{name = String}} |}
      </sample>
    </li>
    <li>
      <p>
        XML Schema <b>element declarations</b> can bound an XML element either
        to a complex type or to a simple type. In the former case the conversion
        is almost identical as what we have seen for complex type conversion.
        The only difference is that this time element's tag must correspond to
        the name of the XML element in the schema element declaration, whereas
        previously it was <code>Any</code> type.
      </p>
      <p>
        In the latter case (element with simple type content), the corresponding
        CDuce types is an element type. Its tag must correspond to the name of
        the XML element in the schema element declaration; its content type its
        the CDuce translation of the simple type provided in the element
        declaration.
      </p>
      <p>
        For example, the following XML Schema element (corresponding to the
        homonymous element defined in <local
          href="manual_schema_samples">mails.xsd</local>):
      </p>
      <sample><![CDATA[
<xsd:element name="header">
 <xsd:complexType>
  <xsd:simpleContent>
   <xsd:extension base="xsd:string">
    <xsd:attribute ref="name" use="required" />
   </xsd:extension>
  </xsd:simpleContent>
 </xsd:complexType>
</xsd:element>
]]></sample>
      <p>
        will be translated to the following CDuce type:
      </p>
      <sample><![CDATA[
# #print_type Mails # header;;
<header {| name = String |}>String
]]></sample>
      <p>
        Note that the type of the element content <em>is not a sequence</em>
        unless the translation of the XML Schema types is a sequence itself (as
        you can notice in the example above). Compare it with the following
        where the element content is not a sequente, but a single record:
      </p>
      <sample><![CDATA[
# #print_type Mails # Date;;
<Date {| |}>{
  positive = Bool;
  year = Int; month = Int; day = Int; hour = Int;
  minute = Int; second = Int;
  timezone =? { positive = Bool; hour = Int; minute = Int }
}
]]></sample>
    </li>
    <li>
      <p>
        XML Schema <b>attribute group definitions</b> are mapped to record types
        containing one field for each attribute declarations contained in the
        group. <tt>use</tt> constraints are respected: optional attributes are
        mapped to optional fields, required attributes to required fields.
      </p>
      <p>
        The following XML Schema attribute group declaration:
      </p>
      <sample><![CDATA[
<xsd:attributeGroup name="mimeTypeAttributes">
 <xsd:attribute name="type" type="mimeTopLevelType" use="required" />
 <xsd:attribute name="subtype" type="xsd:string" use="required" />
</xsd:attributeGroup>
]]></sample>
      <p>
        will thus be mapped to the following CDuce type:
      </p>
      <sample>
# #print_type Mails # mimeTypeAttributes;;
{|  type = [
      'image' | 'text' | 'application' | 'audio' | 'message' | 'multipart' | 'video'
    ];
   subtype = String |}
      </sample>
    </li>
    <li>
      <p>
        XML Schema <b>model group definitions</b> are mapped to CDuce sequence
        types. <tt>minOccurs</tt> and <tt>maxOccurs</tt> constraints are
        respected, using CDuce recursive types to represent <tt>unbounded</tt>
        repetition (i.e. Kleene star).
      </p>
      <p>
        <tt>all</tt> constraints, also known as <em>interleaving
          constraints</em>, can't be expressed in the CDuce type system avoiding
        type sizes explosion. Thus, this kind of content models are normalized
        and considered, in the type system, as sequence types.
      </p>
      <p>
        For a similar reason, <tt>mixed</tt> content models aren't supported by
        CDuce too.
      </p>
      <p>
        As an example, the following XML Schema model group definition:
      </p>
      <sample><![CDATA[
<xsd:group name="attachmentContent">
 <xsd:sequence>
  <xsd:element name="mimetype">
   <xsd:complexType>
    <xsd:attributeGroup ref="mimeTypeAttributes" />
   </xsd:complexType>
  </xsd:element>
  <xsd:element name="content" type="xsd:string" minOccurs="0" />
 </xsd:sequence>
</xsd:group>
]]></sample>
      <p>
        will be mapped to the following CDuce type:
      </p>
      <sample><![CDATA[
# #print_type Mails # attachmentContent;;
[ X1 <content {| |}>String | X1 ] where
X1 = <mimetype {| type = [ ... ]; subtype = String |}>[  ]
]]></sample>
    </li>
  </ul>
</box>

<box title="XML Schema validation" link="validation">
  <p>
    The processes of XML Schema validation and assessment check that an XML
    Schema instance document is valid with respect to an XML Schema document and
    add missing information such as default values. The CDuce's notion of Schema
    validation is a bit different.
  </p>
  <p>
    CDuce permits to have XML values made of arbitrary types, for example you
    can have XML elements which have integer attributes. Still, this feature is
    rarely used because the function used to load XML documents
    (<code>load_xml</code>) returns XML values which have as leaves values of
    type PCDATA.
  </p>
  <p>
    Once you have imported an XML Schema in CDuce, you can use it to validate an
    XML value returned by <code>load_xml</code> against an XML Schema component
    defined in it. The process of validation will basically build a CDuce value
    which has the type corresponding to the conversion of the XML Schema type of
    the component used in validation to a CDuce type. The conversion is the same
    described in the previous secion. Note that is not strictly necessary that
    the input XML value comes from <code>load_xml</code> it's enough that it has
    PCDATA values as leaves.
  </p>
  <p>
    During validation PCDATA strings are parsed to build CDuce values
    corresponding to XML Schema simple types and whitespace are handled as
    specified by XML Schema <code>whiteSpace</code> facet. For example,
    validating the <code>1234567890 </code><em>PCDATA string</em> against an
    <code>xsd:integer</code> simple type will return the CDuce value
    <code>1234567890</code> typed with type <code>Int</code>.<br />
    Default values for missing attributes or elements are also added where
    specified.
  </p>
  <p>
    You can use the <code>validate</code> keyword to perform validation in CDuce
    program. The syntax is as follows:<br /> <code>validate &lt;expr&gt; with
      &lt;schema_ref&gt;</code><br /> where schema_ref is defined as described
    in <a href="#import">XML Schema components import</a>. Same ambiguity rules
    will apply here.
  </p>
  <p>
    More in detail, validation can be applied to different kind of CDuce values
    depending on the type of Schema component used for validation.
  </p>
  <ul>
    <li>
      <p>
        The typical use of validation is to validate against <b>element
          declaration</b>. In such a case validate should be invoked on an XML
        CDuce value as in the following example.
      </p>
      <sample><![CDATA[
# let xml = <Date>"2003-10-15T15:44:01Z" in
  validate xml with Mails # Date;;
  - : <Date {| |}>{
    positive = Bool;
    year = Int; month = Int; day = Int;
    hour = Int; minute = Int; second = Int;
    timezone =? { positive = Bool; hour = Int; minute = Int }
  }
  =
  <Date> {
    positive=`true;
    year=2003; month=10; day=15;
    hour=15; minute=44; second=1;
    timezone={ positive=`true; hour=0; minute=0 }
  }
]]></sample>
      <p>
        The tag of the given element is checked for consistency with the
        element declaration; attributes and content are checked against the
        Schema type declared for the element.
      </p>
    </li>
    <li>
      <p>
        Sometimes you may want to validate an element against an XML Schema
        <b>complex type</b> without having to use element declarations. This
        case is really similar to the previous one with the difference that the
        Schema component you should use is a complex type declaration, you can
        apply such a validation to any XML value. The other important difference
        is that the tag name of the given value is completely ignored.
      </p>
      <p>
        As an example:
      </p>
      <sample><![CDATA[
# let xml = load_xml "envelope.xml" ;;       
val xml : Any = <ignored_tag From="fake@microsoft.com">[
                  <From>[ 'user@unknown.domain.org' ]
                  <To>[ 'user@cduce.org' ]
                  <Date>[ '2003-10-15T15:44:01Z' ]
                  <Subject>[ 'I desperately need XML Schema support in CDuce' ]
                  <header name="Reply-To">[ 'bill@microsoft.com' ]
                  ]
# validate xml with Mails # envelopeType;;
- : <(Any) {| From = String |}>[
      <From {| |}>String <To {| |}>String
      <Date {| |}>{
        positive = Bool;
        year = Int; month = Int; day = Int;
        hour = Int; minute = Int; second = Int;
        timezone =? { positive = Bool; hour = Int; minute = Int }
      }
      <Subject {| |}>String
      <header {| name = String |}>[ String ]* ]
    =
    <ignored_tag From="fake@microsoft.com">[
      <From>[ 'user@unknown.domain.org' ]
      <To>[ 'user@cduce.org' ]
      <Date> {
        positive=`true;
        year=2003; month=10; day=15;
        hour=15; minute=44; second=1;
        timezone={ positive=`true; hour=0; minute=0 }
      }
      <Subject>[ 'I desperately need XML Schema support in CDuce' ]
      <header name="Reply-To">[ "bill@microsoft.com" ]
    ]
]]></sample>
    </li>
    <li>
      <p>
        Similarly you can want to validate against a <b>model group</b>. In this
        case you can validate CDuce's sequences against model groups. Given
        sequences will be considered as content of XML elements.
      </p>
      <p>
        As an example:
      </p>
      <sample><![CDATA[
# let xml = load_xml "attachment.xml";;
  val xml : Any =
    <ignored_tag ignored_attribute="foo">[
      <mimetype type="application"; subtype="msword">[ ]
      <content>[ '\n    ### removed by spamoracle ###\n  ' ]
    ]
# let content = match xml with <_>cont -> cont | _ -> raise "failure";;
  val content : Any = [
    <mimetype type="application"; subtype="msword">[ ]
    <content>[ '\n    ### removed by spamoracle ###\n  ' ]
  ]
# validate content with Mails # attachmentContent;;
- : [ X1 <content {| |}>String | X1 ] where
    X1 = <mimetype {|
          type = [
            'image' | 'text' | 'application' | 'audio' | 'message' | 'multipart' | 'video'
          ];
          subtype = String |}>[  ]
    =
    [ <mimetype type="application"; subtype="msword">[ ]
      <content>[ '\n    ### removed by spamoracle ###\n  ' ]
    ]
]]></sample>
    </li>
<!-- TODO see schema/schema_validator.mli
    <li>
      <p>
        Is also possible to validate CDuce records against <b>attribute
          declarations</b>. If the defined attribute is required, the record is
        scanned for a field having the same name as the attribute. Its content
        is then validated against the simple type associated to the attribute in
        the schema declaration and a new record value is returned. This value is
        identical to the given one except for the content of the validated
        field. Validation fails if no field in the record matches the attribute
        name.
      </p>
      <p>
        If the defined attribute is not required no error is raised if the field
        is missing. If a default value is specified in the attribute declaration
        the returned record will have a corresponding additional field,
        otherwise a record identical to the given one is returned.
      </p>
      <p>
        As an example:
      </p>
      <sample><![CDATA[
# let record = { name = "User-Agent"; added_by = "mutt" };;
  val record : {| name = [ 'User-Agent' ]; added_by = [ 'mutt' ] |}
  =
  { name="User-Agent"; added_by="mutt" }
# validate record with Mails # name ;;
- : { name = String } = { name="User-Agent"; added_by="mutt" }
]]></sample>
    </li>
-->
    <li>
      <p>
        Finally is possible to validate records against <b>attribute groups</b>.
        All required attributes declared in the attribute group should have
        corresponding fields in the given record. The content of each of them is
        validate against the simple type defined for the corresponding attribute
        in the attribute group. Non required fields are added if missing using
        the corresponding default value (if any).
      </p>
      <p>
        As an example:
      </p>
      <sample><![CDATA[
# let record = { type = "image"; subtype = "png" };;
  val record :
    {| type = [ 'image' ]; subtype = [ 'png' ] |} =
      { type="image"; subtype="png" }
# validate record with Mails # mimeTypeAttributes ;;
- : {| type = [ 'image' | 'text' | ... ]; subtype = String |} =
      { type="image"; subtype="png" }
]]></sample>
    </li>
  </ul>
</box>

<box title="XML Schema instances output" link="print_xml">
  <p>
    <b>TODO</b>
  </p>
</box>

</page>