viewcvs.cgi/web/bench.xml

<?xml version="1.0" encoding="ISO-8859-1" standalone="yes"?>
<page name="bench">

<title>Benchmarks</title>

<left>
<p>This page briefly presents preliminary benchmarks on the CDuce
prototype.
</p>
<boxes-toc/>
<p>See also:</p>
<local-links href="index,proto"/>
</left>

<box title="Methodology" link="meth">
<p>
The benchmarks were performed on a Linux box, Athlon 750, 128 Mb RAM.
The results are given in seconds. They were measured with
the Unix <tt>time</tt> command. The user time is reported unless
otherwise specified.
</p>

<p>
These are <b><i>old</i> preliminary benchmarks</b>, and results
should be taken with much care. <b>New benchmarks will be posted soon</b>; 
in the meanwhile you can find them at the end of <a href="http://www.cduce.org/papers/cduce-design.ps.gz">this paper</a>.
</p>
</box>


<box title="CDuce vs. XSLT" link="xslt">

<p>
We performed benchmarks between CDuce and a fast XSLT processor (the
<tt>xsltproc</tt> program from <a href="http://xmlsoft.org/XSLT/">
Gnome libxslt library</a>).
</p>

<p>
The example is intended to evaluate run-time efficiency. Even though
the transformation is quite complex, type-checking time for CDuce 
is negligible. A few remarks:
</p>
<ul>
<li>type-based optimization
brings almost no gain (only tag checks can be avoided);</li>
<li>the CDuce program must validate its input document (the XSLT
stylesheet can operate on any document);</li>
<li>the XSLT processor is written in C; the CDuce interpreter
is written in Objective Caml.</li>
</ul>

<p>
Taking these remarks into account, the results below are more than
encouraging.
</p>

<p>
<a href="benchmarks/split.cd">CDuce source</a> -
<a href="benchmarks/split.xslt1.xml">XSLT source</a> -
<a href="benchmarks/split.xslt2.xml">XSLT source (another version)</a> -
<a href="benchmarks/split.ml">OCaml file</a> (to generate random documents).
</p>


<table border="1">
 <tr> <td>split</td> 
      <th>XSLT (1)</th> 
      <th>XSLT (2)</th> 
      <th>CDuce</th> </tr>

 <tr align="right"> <th>5 Mb</th> 
      <td>8.90</td> <td>8.05</td> <td>5.30</td> </tr>
 <tr align="right"> <th>27 Mb</th>
      <td>47.10</td> <td>68.40</td> <td>33.85</td> </tr>
</table>

<p>
The table above displays CPU user time (parsing + transformation) .
The table below, instead, displays total wall-clock time
(including system time). The results indicate that CDuce
behave much better when the XML document
does'nt fit in memory (and swapping occurs).
</p>

<table border="1">
 <tr> <td>split</td> 
      <th>XSLT (1)</th> 
      <th>XSLT (2)</th> 
      <th>CDuce</th> </tr>

 <tr align="right"> <th>5 Mb</th> 
      <td>9.05</td> <td>8.26</td> <td>5.40</td> </tr>
 <tr align="right"> <th>27 Mb</th>
      <td>320.80</td> <td>324.50</td> <td>83.65</td> </tr>
</table>

</box>

<box title="CDuce vs. XDuce" link="xduce">

<section title="Addrbook">

<p>
This example was taken from XDuce distribution.
The transformation is simple;
the example is intended to evaluate run-time efficiency.
</p>

<p>
<a href="benchmarks/addrbook.cd">CDuce source</a> -
<a href="benchmarks/addrbook.q">XDuce source</a> -
<a href="benchmarks/addrbook.ml">OCaml file</a> (to generate random documents).
</p>


<table border="1">
 <tr> <td>Addrbook</td> 
      <th>XDuce 0.2.4</th> 
      <th>XDuce 0.2.4 + patopt</th> 
      <th>XDuce 0.4.0 + patopt</th> 
      <th>CDuce</th> </tr>

 <tr align="right"> <th>Type-checking</th> 
      <td>0.04</td> <td>0.05</td> <td>0.15</td> <td>0.04</td> </tr>
 <tr align="right"> <th>Small file (60 Ko)</th>
      <td>1.56</td> <td>0.14</td> <td>9.6</td> <td>0.09</td> </tr>
 <tr align="right"> <th>Medium file (600 Ko)</th>
      <td>260</td> <td>1.16</td> <td>*</td> <td>0.46</td> </tr>
 <tr align ="right"> <th>Large file (6 Mb)</th>
      <td>*</td> <td>31.05</td> <td>*</td> <td>6.37</td> </tr>

</table>

<p>
The * corresponds to checks stopped after ten minutes.
</p>
</section>

<section title="Html2latex">

<p>
This example was taken from XDuce distribution and simplified
(removed support for tables). The transformation is more complex;
the example is intended to evaluate type-checking time.
</p>

<p>
<a href="benchmarks/html2latex.cd">CDuce source</a> -
<a href="benchmarks/xhtml-trans.cd">CDuce source (XHTML Transitional DTD)</a> -
<a href="benchmarks/html2latex.q">XDuce source</a> -
<a href="benchmarks/tst_html2latex.xml">Sample input XML file</a>.
</p>

<table border="1">
 <tr> <td>Html2latex</td> 
      <th>XDuce 0.2.4</th> 
      <th>XDuce 0.2.4 + patopt</th> 
      <th>XDuce 0.4.0 + patopt</th> 
      <th>CDuce</th> </tr>

 <tr align="right"> <th>Type-checking</th> 
      <td>1.5</td> <td>21.4</td> <td>9.5</td> <td>0.4</td> </tr>
 <tr align="right"> <th>Total</th> 
      <td>2.3</td> <td>26.5</td> <td>10.0</td> <td>0.8</td> </tr>
 </table>

</section>

</box>


<box title="CDuce vs. XQuery" link="xquery">

<p>
<b>A benchmark test with the <a href="http://www.w3.org/TR/xmlquery-use-cases"> XQuery Use Cases</a> is in preparation</b>
</p>
</box>
</page>
1	<?xml version="1.0" encoding="ISO-8859-1" standalone="yes"?>
2	<page name="bench">
3
4	<title>Benchmarks</title>
5
6	<left>
7	<p>This page briefly presents preliminary benchmarks on the CDuce
8	prototype.
9	</p>
10	<boxes-toc/>
11	<p>See also:</p>
12	<local-links href="index,proto"/>
13	</left>
14
15	<box title="Methodology" link="meth">
16	<p>
17	The benchmarks were performed on a Linux box, Athlon 750, 128 Mb RAM.
18	The results are given in seconds. They were measured with
19	the Unix <tt>time</tt> command. The user time is reported unless
20	otherwise specified.
21	</p>
22
23	<p>
24	These are <b><i>old</i> preliminary benchmarks</b>, and results
25	should be taken with much care. <b>New benchmarks will be posted soon</b>;
26	in the meanwhile you can find them at the end of <a href="http://www.cduce.org/papers/cduce-design.ps.gz">this paper</a>.
27	</p>
28	</box>
29
30
31	<box title="CDuce vs. XSLT" link="xslt">
32
33	<p>
34	We performed benchmarks between CDuce and a fast XSLT processor (the
35	<tt>xsltproc</tt> program from <a href="http://xmlsoft.org/XSLT/">
36	Gnome libxslt library</a>).
37	</p>
38
39	<p>
40	The example is intended to evaluate run-time efficiency. Even though
41	the transformation is quite complex, type-checking time for CDuce
42	is negligible. A few remarks:
43	</p>
44	<ul>
45	<li>type-based optimization
46	brings almost no gain (only tag checks can be avoided);</li>
47	<li>the CDuce program must validate its input document (the XSLT
48	stylesheet can operate on any document);</li>
49	<li>the XSLT processor is written in C; the CDuce interpreter
50	is written in Objective Caml.</li>
51	</ul>
52
53	<p>
54	Taking these remarks into account, the results below are more than
55	encouraging.
56	</p>
57
58	<p>
59	<a href="benchmarks/split.cd">CDuce source</a> -
60	<a href="benchmarks/split.xslt1.xml">XSLT source</a> -
61	<a href="benchmarks/split.xslt2.xml">XSLT source (another version)</a> -
62	<a href="benchmarks/split.ml">OCaml file</a> (to generate random documents).
63	</p>
64
65
66	<table border="1">
67	<tr> <td>split</td>
68	<th>XSLT (1)</th>
69	<th>XSLT (2)</th>
70	<th>CDuce</th> </tr>
71
72	<tr align="right"> <th>5 Mb</th>
73	<td>8.90</td> <td>8.05</td> <td>5.30</td> </tr>
74	<tr align="right"> <th>27 Mb</th>
75	<td>47.10</td> <td>68.40</td> <td>33.85</td> </tr>
76	</table>
77
78	<p>
79	The table above displays CPU user time (parsing + transformation) .
80	The table below, instead, displays total wall-clock time
81	(including system time). The results indicate that CDuce
82	behave much better when the XML document
83	does'nt fit in memory (and swapping occurs).
84	</p>
85
86	<table border="1">
87	<tr> <td>split</td>
88	<th>XSLT (1)</th>
89	<th>XSLT (2)</th>
90	<th>CDuce</th> </tr>
91
92	<tr align="right"> <th>5 Mb</th>
93	<td>9.05</td> <td>8.26</td> <td>5.40</td> </tr>
94	<tr align="right"> <th>27 Mb</th>
95	<td>320.80</td> <td>324.50</td> <td>83.65</td> </tr>
96	</table>
97
98	</box>
99
100	<box title="CDuce vs. XDuce" link="xduce">
101
102	<section title="Addrbook">
103
104	<p>
105	This example was taken from XDuce distribution.
106	The transformation is simple;
107	the example is intended to evaluate run-time efficiency.
108	</p>
109
110	<p>
111	<a href="benchmarks/addrbook.cd">CDuce source</a> -
112	<a href="benchmarks/addrbook.q">XDuce source</a> -
113	<a href="benchmarks/addrbook.ml">OCaml file</a> (to generate random documents).
114	</p>
115
116
117	<table border="1">
118	<tr> <td>Addrbook</td>
119	<th>XDuce 0.2.4</th>
120	<th>XDuce 0.2.4 + patopt</th>
121	<th>XDuce 0.4.0 + patopt</th>
122	<th>CDuce</th> </tr>
123
124	<tr align="right"> <th>Type-checking</th>
125	<td>0.04</td> <td>0.05</td> <td>0.15</td> <td>0.04</td> </tr>
126	<tr align="right"> <th>Small file (60 Ko)</th>
127	<td>1.56</td> <td>0.14</td> <td>9.6</td> <td>0.09</td> </tr>
128	<tr align="right"> <th>Medium file (600 Ko)</th>
129	<td>260</td> <td>1.16</td> <td>*</td> <td>0.46</td> </tr>
130	<tr align ="right"> <th>Large file (6 Mb)</th>
131	<td></td> <td>31.05</td> <td></td> <td>6.37</td> </tr>
132
133	</table>
134
135	<p>
136	The * corresponds to checks stopped after ten minutes.
137	</p>
138	</section>
139
140	<section title="Html2latex">
141
142	<p>
143	This example was taken from XDuce distribution and simplified
144	(removed support for tables). The transformation is more complex;
145	the example is intended to evaluate type-checking time.
146	</p>
147
148	<p>
149	<a href="benchmarks/html2latex.cd">CDuce source</a> -
150	<a href="benchmarks/xhtml-trans.cd">CDuce source (XHTML Transitional DTD)</a> -
151	<a href="benchmarks/html2latex.q">XDuce source</a> -
152	<a href="benchmarks/tst_html2latex.xml">Sample input XML file</a>.
153	</p>
154
155	<table border="1">
156	<tr> <td>Html2latex</td>
157	<th>XDuce 0.2.4</th>
158	<th>XDuce 0.2.4 + patopt</th>
159	<th>XDuce 0.4.0 + patopt</th>
160	<th>CDuce</th> </tr>
161
162	<tr align="right"> <th>Type-checking</th>
163	<td>1.5</td> <td>21.4</td> <td>9.5</td> <td>0.4</td> </tr>
164	<tr align="right"> <th>Total</th>
165	<td>2.3</td> <td>26.5</td> <td>10.0</td> <td>0.8</td> </tr>
166	</table>
167
168	</section>
169
170	</box>
171
172
173	<box title="CDuce vs. XQuery" link="xquery">
174
175	<p>
176	<b>A benchmark test with the <a href="http://www.w3.org/TR/xmlquery-use-cases"> XQuery Use Cases</a> is in preparation</b>
177	</p>
178	</box>
179	</page>