diff options
author | Lars Knoll <lars.knoll@nokia.com> | 2009-03-23 09:34:13 (GMT) |
---|---|---|
committer | Simon Hausmann <simon.hausmann@nokia.com> | 2009-03-23 09:34:13 (GMT) |
commit | 67ad0519fd165acee4a4d2a94fa502e9e4847bd0 (patch) | |
tree | 1dbf50b3dff8d5ca7e9344733968c72704eb15ff /tests/auto/xmlpatterns/XSLTTODO | |
download | Qt-67ad0519fd165acee4a4d2a94fa502e9e4847bd0.zip Qt-67ad0519fd165acee4a4d2a94fa502e9e4847bd0.tar.gz Qt-67ad0519fd165acee4a4d2a94fa502e9e4847bd0.tar.bz2 |
Long live Qt!
Diffstat (limited to 'tests/auto/xmlpatterns/XSLTTODO')
-rw-r--r-- | tests/auto/xmlpatterns/XSLTTODO | 1450 |
1 files changed, 1450 insertions, 0 deletions
diff --git a/tests/auto/xmlpatterns/XSLTTODO b/tests/auto/xmlpatterns/XSLTTODO new file mode 100644 index 0000000..5a20441 --- /dev/null +++ b/tests/auto/xmlpatterns/XSLTTODO @@ -0,0 +1,1450 @@ +This is a TODO file for XSL-T 2.0 support. + +- LHF: + * Warning bug, last parameter is always whined about. + * Box in comment/PI/text/ws(?) handling -- pending Matthias + * type036 -- namespace on top element isn't copied + * XTDE0865 + * Attend XSLTTokenizer::isXSLT() + * Remove redundant set() calls in setFocusHelper(). + +- Missing features: + General Priority + --------------------- + * 1.0 QXmlQuery::evaluateTo(QIODevice *) P1 DONE + * 1.0 Test suite integration P1 DONE + * 1.0 xsl:key P1 + * 1.0 fn:key() P1 + * 1.0 2.0 Compatibility mode P1 + * 1.0 Regular parameters in templates P1 + * 1.0 xsl:include P1 + * 1.0 xsl:copy-of P1 + * 1.0 xsl:copy P1 + * 1.0 xsl:import P1 + * 1.0 fn:format-number P1 + * 1.0 xsl:message P2 + * 1.0 fn:current() P1 DONE + * 2.0 fn:type-available() P3 DONE + * 2.0 xsl:use-when P3 + * 2.0 fn:unparsed-entity-uri() P3 + * 2.0 fn:unparsed-entity-public-id() P3 + * 2.0 Tunnel Parameters P3 + * 2.0 xsl:attribute-set P3 + * 1.0 xsl:decimal-format P2 + * 1.0 xmlpatterns: initial template P1 DONE + * 1.0 xsl:number P1 + * 1.0 Complete handling of xsl:sort P2 + * 2.0 Grouping + - fn:current-group() + - fn:grouping-key() + - xsl:for-each-group() + * 2.0 Regexp + - xsl:analyze-string + - xsl:matching-substring + - xsl:non-matching-substring + - fn:regex-group() + * 2.0 Date & Time formatting + - fn:format-dateTime() + - fn:format-date() + - fn:format-time() + + Serialization & Output: + ---------------------- + * 1.0 xsl:output + --- Tie together serialization. Should we add + QXmlQuery::evaluateTo(QIODevice 1.0 const) ? + * 2.0 xsl:character-maps + * 2.0 xsl:character-map + * 2.0 xsl:result-document + --- Should the "default output" be handle with xsl:result-document? Would + depend on compilation. + +Optimizations: + * Remove adjacent text node constructors + * Remove string-join when first arg's static cardinality is not more than one + * Remove string-join when the second arg is statically known to be the empty string. + * Remove string-join when the second arg is a single space and the parent is a text node ctor. + * Rewrite to operand if operands are one. What about type conversions? + * Replace lookups with xml:id with calls on id(). + * Recognize that a/(b, c) is equal to a/(b | c). The later does selection and node sorting in one step. + * Remove LetClause which has empty sequence as return clause, or no variable dependencies at all. + * Do a mega test for rewriting /patterns/: + "node() | element()" => element() + "comment() | node()" => comment() + + and so forth. This sometimes happens in poorly written patterns. How does + this rewrite affect priority calculation? + +Tests: + * xml:id + - Come on, the stuff needs to be reorganized xml:id. + - Read in xml:id document with whitespace in attrs, write the doc out. Attrs should be normalized. + - Do lookups of IDs with xml:id attrs containing whitespace. + + * current() + - Use current() inside each instruction + - In a template pattern + - Several invocations: current()/current()/current() + + + * Diagnosticsts: + - See http://www.w3.org/Bugs/Public/show_bug.cgi?id=5643 . Comments + should be taken into account when comparing. This suggests that we + don't have any test which produces a document with XML comments. + + * element-available() + - Review the tests. + - Try using declarations in XSL-T, should return false + - Use xsl:variable(both instr and decl) + - invoke with all the XSL-T instructions. + - Should return false for when, otherwise, matching-substring, non-matching-substring, etc? + - Supply the namespace in the name via the default namespace, no prefix. + + * unparsed-text() + - Load an empty file + - Use a fragment in the URI + - Use an invalid URI + - Use device bindings and a QRC to ensure that we're not using a generic + network manager. + - Duplicate all the network tests. Same as for doc() + + * unparsed-text-available() + - Same as for unparsed-text() + + * Sequence constructor that contains only: + - XML comment + - whitespace text node + - processing instruction + - a mix of the three + + * xsl:function + - Ensure that it's not it's not in scope for use-when. + - xsl:function/xsl:param: use processing instructions, whitespace and comments as child: should be stripped + - Use <xsl:function/> : @name missing. + - Don't strip ws, and have ws between two xsl:param, and between xsl:function and xsl:param. + - Use xsl:function with no body. + - use xsl:param/@tunnel = no + - use xsl:param/@tunnel = yes + - use an invalid value for xsl:param/@tunnel = yes + - Have a non-WS text node in xsl:function/xsl:param/ + - Have a WS text node in xsl:function/xsl:param/ + - Have a WS text node in xsl:function/xsl:param/ while preserving WS. + - use a comment as child of xsl:param + - use a PI as child of xsl:param + - XTSE0770 with import precedence and all that. + - have two identical function in the stylesheet. The last has override=no. Should still report XTSE0770. + - have @override with invalid value. + - have whitespace inside xsl:param with different strip modes. + - Have @select => error + - Have body => error + - call current() inside body. XPDY0002? + + * Does xml:base/StaticBaseURI and StaticCompatiblityStore prevent proper + type checking due to expectedOperandTypes() returns item()*? + + * xsl:template/xsl:param + - Have @required=yes, and have @select => error + - Have @required=yes, and have body => error + - Have a variable reference in a template after another, which has + param, to ensure they aren't in scope. + + * xsl:template/@match + - Have a pattern with unions, and have a body which relies on its + static type. + + * @version: + Have @version on *all* attributes. + + * xsl:call-template + - Have a variable reference just after a xsl:call-template which has + with-param, to ensure they aren't in scope. + - Have an xsl:with-param which isn't used in the template. Error? + - Have an xsl:with-param that has a type error. + - an xsl:with-param is not in scope for the next one. Test this => error. + - Have a call:template, whose with-param computes its value by calling + another template, while using an with-param too. + + * XQuery: + - DONE Ensure namespace {expr} {expr} is flagged as invalid + - Use all XSL-T functions: error. Or we do that already? + - Ensure order by collation 1 + 1 is an error + - Ensure order by collation {1 + 1} is an error + + * document() + - Basic node deduplication, no test exists for that. + + * xsl:perform-sort + - Have no xsl:sort. Error. Must be at least one. + - have xsl:sort with invalid value. + - sort atomic values. + - Trigger "The stable attribute is permitted only on the first xsl:sort element within a sort key specification" + - have xsl:sort with no select and no seq ctor. + - trigger the delegated queueing. All instructions inside.. xsl:sort? + - have multiple sort statements, with the last being <xsl:sort/> only. + - have WS between xsl:sort that is not ignorable. + - Use a variable reference whose name is equal to our synthetic name. This should be XPST0008, but probably isn't. + - Have an invalid value in xsl:sort/order. Use space + - have xsl:sort return numbers, but data-type specify string. + - have an AVT in xsl:sort/@lang + - have an AVT in xsl:sort/@case-order + - have an AVT in xsl:sort/@data-type + - have an AVT in xsl:sort/@stable + - Have mixed result, and hence incorrectly trigger XPTY0018 which the code currently raise. + - Depend on the context position inside xsl:sort, when being child of + perform-sort. Currently we create singleton focuses(I think), while + we want the focus to be over the whole input sequence, not on indivual items. + - Have <xsl:perform-sort select="valid-expr"/>: xsl:sort is missing + - Use current() in the xsl:sort and the body, to ensure the right scope is picked up + + * xsl:copy-of + - Have a text node. It's not allowed. + - Have PIs, comments, and ignorable whitespace as children. Sigh. + + * xsl:namespace + - Use xsl:fallback. + - Use xsl:namespace inside xsl:variable and introspec the result in various + ways. This is a big area, we don't have namespace nodes in XQuery. Yes, calling evaluateSingleton() will probably crash. + - Use no select and no body, error: XTSE0910 + - Have name expression evaluate to the empty sequence. + + * Sequence ctor that: + - Has invalid element in XSL namespace. E.g, xsl:foo + + * xsl:import + - Have element as child as xsl:import: disallowed. + - Have text as child as xsl:import: disallowed. + - Have PIs and comments as child as xsl:import: allowed. + + * xsl:include + - Have element as child as xsl:include: disallowed. + - Have text as child as xsl:include: disallowed. + - Have PIs and comments as child as xsl:include: allowed. + + * xsl:strip-space + - Have PIs, comments, whitespace as child. + + * xsl:element + - Extract EBV from result. + - Use space in validation element. + + * xsl:perform-sort + - Have PIs and comments in between xsl:sort elements. + + * xml:space + - We never pop our stack. Fix the bug, and ensure we have tests for it. + + * fn:unparsed-entity-uri + - Check type of return value + - Do basic unparsed-entity-uri("does-not-exist") + + * fn:unparsed-entity-public-id + - Do basic unparsed-entity-uri("does-not-exist"), two permutations, check the spec + + * xsl:element + - Use disallowed attribute: select + - use unknown type in @type + - Use @namespace, but be not in the lexical space of xs:anyURI + - use disallowed enumeration in @validation + - have a name expression that evaluates to a xs:QName value as opposed to a string. + - have a name expression that evaluates to a xs:QName value as opposed to a string. but + also have the namespace attribute + + * xsl:attribute + - Use disallowed attribute: match + - use unknown type in @type + - Use @namespace, but be not in the lexical space of xs:anyURI + - use disallowed enumeration in @validation + - have a name expression that evaluates to a xs:QName value as opposed to a string. + - have a name expression that evaluates to a xs:QName value as opposed to a string. but + also have the namespace attribute + + * xsl:template + - Use the union keyword, it's forbidden, only "|" is allowed + - Use an expression other than Literal and VarRef in KeyValue[8] + - use a function other than key(). + - have a declaration that only can apperar as a child of xsl:stylesheet. + - Have an element in the XSL-T namespace, but which is invalid, e.g "bar" + - Use an axis other than child or attribute in pattern. + - Have a template that no no match and no name attribute., XTSE0500 + - use child::document-node() in pattern + - use @foo/child in pattern + - apply templates to parentless attributes. + - Have 3e3 in @priority + - Have a @match with more than two alternatives, e.g "a | b | c", and have them all actually matching. + - Use an XML name in the mode so we trigger + NCNameConstructor::validateTargetName() + - A template which only has a non-WS text node. + - A template with param, followed by text node. + + * Simplified stylesheet + - Use @version attribute only on doc element. Should fail, since @{XSL-T]version must be present + + * fn:current() + - Have <xsl:value-of select="current()"/> + + * xsl:variable have a variable reference appearing before its global declaration, and then somehow trigger recursion. + * xsl:choose + - elements/nodes intermixed with xsl:choose/xsl:when + - invalid attribute on xsl:choose + - invalid attribute on xsl:when + - invalid attribute on xsl:otherwise + - invalid attribute on xsl:if + - invalid attribute on xsl:template + - invalid attribute on xsl:stylesheet + - invalid attribute on xsl:transform + - xsl:otherwise in the middle between xsl:when elements. + - use namespace declarations on xsl:when + - use namespace declarations on xsl:otherwise + - use namespace declarations on xsl:choose + + * Namespaces: + - Have: + <xsl:sequence xmlns:bar="http://example.com/" select="1"/> + <xsl:sequence select="bar:foo()"/> + + * XPath + - For each XQuery-specific expression, add a test using that expression: + - typeswitch + - let + - validate + - extension expression + - unordered + - ordered + - for + - computed text node constructor + - computed attribute constructor + - computed comment constructor + - computed PI constructor + - computed element constructor + - computed document constructor + - direct element constructor + - direct comment constructor + - direct PI constructor + - all declarations + + - Use all the predefined prefixes in XQuery; non are in XSL-T. + + * xsl:when + - Use xml:space on it + + * xsl:otherwise + - Use xml:space on it + + * xsl:version + - Use letters, XTSE0110 + - Use a float: 2e3, XTSE0110 + - Use a weird number, 2.00000001 + + * xsl:document + - use disallowed attribute: select. + - use unknown type in @type + - use disallowed enumeration in @validation + - What happens if the type in @type is unknown? + - Use xml:base attr and check URI. + + * xsl:sequence + - use match attribute + + * xsl:stylesheet + - Use @xsl:default-collation on xsl:stylesheet. Shouldn't have any effect. Or? + - Use an XSL-T instruction as child -- invalid. + - Have an element in the XSL-T namespace, but which is invalid, e.g "foo" + - Have xsl:default-collation="http://example.com/" on xsl:stylesheet + - Use prefix local: in say a function name. Not allowed. + - Use comments after document element. + - XTSE0010: <xsl:invalid version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"/> + - Change the version with @xsl:version on all elements that we have. + + * Patterns. + - Basic */* test: + <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"><xsl:template match="*/*"><xsl:sequence select="'MATCH'"/></xsl:template> + + +</xsl:stylesheet> + + - Basic a/b test: + <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"><xsl:template match="a/b"><xsl:sequence select="'MATCH'"/></xsl:template> + + +</xsl:stylesheet> + * xsl:strip-whitespace + - Use a namespace prefix which is not unboudn + - have a syntax error in one of the node tests + + * xsl:preserve-whitespace + - Use a namespace prefix which is not unboudn + - have a syntax error in one of the node tests + + * xsl:value-of + - select attribute, and comment in body(error, XTSE0870) + - select attribute, and processing instruction in body(error, XTSE0870) + - select attribute, and CCDATA in body(error, XTSE0870) + - select attribute, and element in body(error, XTSE0870) + - use xsl:sequence in body. Default separator should be none. + - use match attribute + - use double apostrophes/quotes. How are they dealt with? + + * xsl:apply-templates + - use match attribute + - apply in a mode for which no templates are declared + - apply in a mode which is mispelled for another. + - Have: <xsl:apply-templates select="id('id2')/a | id('id5')"/> + We CRASH + + * xsl:for-each + - No body: <xsl:for-each select="abc"/> + - No select attribute: <xsl:for-each>text</xsl:for-each> + - Have mixed result, and hence incorrectly trigger XPTY0018 which the code currently raise. + - Have: + <xsl:for-each select="1, 'asd'"> + <xsl:sequence select="."/> + </xsl:for-each> + + * xsl:variable + - Test that function conversion rules are invoked + - For what is an xsl:variable in scope? Where does the spec state it? Test + that it is not in scope where applicable. + - Have: <variable name="a" select=""/> + + * xsl:text + - count the result of a template that has text node(non-ws), + xsl:text(content), xsl:content(zero content), text node(non-ws + - Have an element inside xsl:text: XTSE0010. + - Use comments and PIs intermixed with text inside. + + * xsl:for-each + - use match attribute + - What should this produce? Saxon produces "123" but with xsl:text removed, "1 2 3". + <xsl:template match="/"> + <xsl:for-each select="1 to 3"> + <xsl:sequence select="."/> + <xsl:text></xsl:text> + </xsl:for-each> + </xsl:template> + + * xsl:if + - Have <xsl:if test="">body</xsl:if>. Error + - Have <xsl:if test="valid-test"/>. That is, empty body. + + * xsl:sequence + - select attribute missing: <xsl:sequence/> + - content other than xsl:fallback, e.g text node. + - How do we sort? + + * for every state for XSL-T parsing: + - Use invalid element that is in the XSL-T namespace. + + * In all cases expressions are queued/generated: + - Trigger expression precedence bugs, due to lack of paranteses. + + * Use xml:space in stylsheeet that doesn't have value preserve nor default. + * For each case we have while(!reader.atEnd()): + - test that triggers parser error and that we detect it properly. + + * for every element that allows text: + * Use CDATA. QXmlStreamReader distinguishes between the two. text before and after.:wa + * Use XML Comments and split up text nodes. + + * Patterns: + * Ensure node() doesn't match document nodes(). + * "//" is an invalid pattern + * Is there some expression which has no children? findAxisStep() + * Use @*/asdo + * XPST0003: key(concat("abc", "def"), "abc") + * XPST0003: id(concat("abc", "def")) + * XPST0003: concat('abc', 'def') // WILL CRASH + * XPST0003: unknownFunction() + * Use double: key("abc", 3e3) + * Use three argument key() in pattern. + + * Simplified stylsheet modules: + * Omit the xsl:version attribute. XTSE0010 + + * type-available() + * We have no tests at all? + + * have xml:base on the following elements and check them with + static-base-uri(): + - all instructions + - all declarations, eg: + - xsl:choose, xsl:choice, xsl:otherwise + - xsl:template + - xsl:function + - etc + + Embedded stylesheet modules + - Verify that we don't choke on xsl attrs with invalid attributes outside; + "In an embedded stylesheet module, standard attributes appearing on + ancestors of the outermost element of the stylesheet module have no effect." + + Parsing: + - Error message for: + <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> + <xsl:template match="/"> + <e/> + </xsl:stylesheet> + - Use the document "<do/" as focus. + - Write a test for each call to checkForParseError(); + + + function-available: + - Generally low coverage it seems. + + +<xsl:template match="/"/> <!-- t0 --> +<xsl:template match="*"/> <!-- t1 --> +<xsl:template match="asd"/> <!-- t2 --> +<xsl:template match="comment()"/> <!-- t3 --> +<xsl:template match="a/b"/> <!-- t4 --> + +<xsl:apply-templates select="*"/> + + +*( +((/)/call-template(t0)) +(*/call-template(t1)) +(element(asd)/call-template(t2)) +(comment()/call-template(t3)) +(a/b/call-template(t3)) +) + +==> + +*/typeswitch(.) + case $g0 as document-root() return call-template(t0) + case $g0 as element() return call-template(t1) + case $g0 as element(asd) return call-template(t2) + case $g0 as comment() return (call-template(t3) + case $g0 as a/b return (call-template(t4) + + +Patterns are used in: + - xsl:for-each-group/@group-starting-with + - xsl:key/@match + - xsl:number/(@count, @from) + - xsl:template/@match + + + +c/b +=> +child-or-self::element(b)[parent::element(c)] + +c/b/a +=> +child-or-self::element(a)[parent::element(b)[parent::element(c)]] + +d/c/b/a +=> +child-or-self::element(a)[parent::element(b)[parent::element(c)[parent::element(d)]]] + + + +----------------------------------- + <xsl:apply-templates select="foo"/> + => + + child::element(foo) map apply-template(#default) +----------------------------------- + +----------------------------------- + <xsl:apply-templates mode="yo" select="foo"> + <xsl:sort select="@bar"/> + </xsl:apply-templates> + => + + let $g0 := for $g1 in child::element(foo) + order by @bar + return $g1 + return apply-template(yo) +----------------------------------- + +----------------------------------- +<xsl:perform-sort select="$in"> + <xsl:sort select="@sortKey"/> +</xsl:perform-sort> + + => + +sort $in/order by @sortKey +----------------------------------- + + +----------- + +John Snelson of Oracle Berkeley DB XML & XQilla writes in private mail: + + I'd had the same thought myself, too - you must be reading my mind ;-) + +What is he referring to? + +If one spends some time on fancy diagrams, QtXmlPatterns[LINK]'s, the XQuery +engine that shipped in Qt 4.4, architecture looks like this: + + +Recently I've started implementing XSL-T 2.0(hopefully to be done for Qt 4.5) +and the whole approach to this is modifying the existing codebase as follows: + + + + + + +Put differently, when QtXmlPatterns is dealing with XSL-T stylesheets, it +replaces the XQuery tokenizer with a tokenizer which translates the +stylesheet into XQuery tokens, that is consumed by the existing XQuery +parser, extended with both grammar non-terminals and tokens to accomodate the +XSL-T features that XQuery doesn't have. In compiler terms, it can be seen as +an "extreme" frontend. Not only is the same intermediate representation used, +the grammar is too. + +What is the point of this? + +The functional overlaps XQuery, XSL-T and others as well have is of course +widely established. Even the specifications are at times generated from the +same source documents, and that implementations subsequently modularize code +is of course second nature to any engineer, and seen to some degree or +another in contemporary implementations. Typically this happens in a +traditional fashion, classes are re-used, their functionality widened to +cover both/more languages. However, I believe doing it directly on the +grammar level introduce several advantages. + +The parser is based on Bison and since it's not running in the experimental +pull mode, it uninterruptedly calls the tokenizer. The tokenizer, class +XSLTTokenizer, in turns calls an pull-based XML parser: QXmlStreamReader. +What often complicate in ocassions like this is who that gets the right to +call who, and who gets the convenience of tracking state in a natural way +through a call stack. + +XSLTTokenizer is conveniently implemented: as it encounters declarations and +instructions in the stylsheet, it recursively descends in the XSL-T grammar +through its own functions, adding tokens to a queue, which is delivered to +the parser when asked -- and when the queue is empty it resumes queuing +tokens. The tokenizer is fairly crude, it queues up tokens for instructions +uninterrupted, and only have states between declarations. Hence, +XSLTTokenizer queues up tokens for each template and function body, but +enters "delivery mode" inbetween. This of course periodically breaks +streaming since it's buffering up tokens, but considering that the memory +usage for tokens is low and that a finer granularity for states(say, being +able to pop the stacks when being inbetween two xsl:when elements) requires a +significant effort, this is fine until proven otherwise. + + +Advantages +--------------- +discuss analysis. + + +XSLTTokenizer rewrite XSL-T to XQuery as follows:' + +Instructions +------------- +xsl:if An if/then/else expression whose else branch is the empty sequence + +xsl:choose: again, a nesting of if/then/else expressions + +xsl:value-of: a computed text node constructor. Its body contains a call to +string-join() involving the separator attribute + +xsl:variable: a let/return binding. Since XSL-T is statement-like in its +sequence constructors, parantheses are used to ensure the variable binding is +in-scope for all subsequent statements. + +for-each: it is the iteration/mapping mechanism XQuery fails to supply, +despite path steps and the FLWOR machinery. for-each iterates using a +focus(which for doesn't, but paths do), but can do so over atomic values and +unconditionally without sorting the result by document order(which paths +can't/doesn't, but for do). For implementations that normalize paths into for +loops as the formal semantics do, the approach is straight forward. In +QtXmlPatterns' case, a synthetic token is queued which signals to create +a "relaxed" path expression which skips halting on atomic values in its +operands(XPTY0019) and also skips node sorting. + +All "direct" node constructors, like <myElement/>, and "computed" node +constructors, like xsl:element, are all rewritten into the corresponding +XQuery computed node constructors. In some cases direct node constructors +could have been used, but in anycase the IR yielded is the same, and that +computed constructors happen to use less tokens. + +A particular case is xsl:namespace, an instruction which doesn't have any +corresponding expression in XQuery. In the case of QtXmlPatterns, the code +obvious already have a notion of "create this namespace on this element", and +an AST node was trivially added for fetching the namespace components +computationally. However, the introduction of xsl:namespace in an XQuery +implementation is not to be taken lightly wrt. to for instance testing, since +it introduces a new node type. + +perform-sort: surprisingly this expression of all complicate matters, for the +simple reason that its operands occur in the opposite order compared to +XQuery when the input sequence is supplied through a sequence constructor, +hence breaking the streamed approach. XSLTokenizer solves this by +buffer: the attributes of the xsl:perform-sort elements are stored, +the xsl:sort elements queued onto a temporary queue, and subsequently is +either the select attribute or the sequence constructor queued, and the +tokens for xsl:sort appended afterwards. This complicated code greatly, since +XSLTokenizer had to be able to "move around" sequences of tokens. + +In addition perform-sort has the same problem as for-each, the iteration +mechanism falls inbetween paths and the for loop. The focus for perform-sort +is also the focus for the sequence constructor and the select attribute, but +the focus for the xsl:sort elements is the initial sequence. This is +approached by having a for loop, and where the expression in each order by +clause has a relaxed path expression whose left operand is a variable +reference to what the for loop bound. +TODO Doesn't work. Focus size wrong. + +This is an approach that implementations of the "second generation" of the +technologies can take. The bif difference is that XSL-T 2.0 doesn't have the +restrictions of 1.0, more evident in XQuery's syntax. + +xsl:sort XSL-T is much more dynamic than XQuery through the use of templates, +but also +because more decisions can be taken at runtime through all attribute value +templates. xsl:sort is surely a good example of this with its AVTs for +language, order, collation, stability and what not. XQuery's order by stands +in strong contrast, which has these coded in the grammar. In QtXmlPatterns' +case, the AST node corresponding to order by was generalized to take things +such as stability and order from operands. This is paid by the code paths in +XQuery since for them are constants generated and inserted as operands even +though its known at compile time what is needed. However, considering that +these evaluations are not inside the actual sort loop, but instead only +computed on each sort invocation, it shouldn't be too bad. + +xsl:message + +Templates +------------------------- + +A big bucket of questions for an XQuery implementation is of course the +introduction of templates. In this case it is too of large interest to +rewrite relevant code into primitive XQuery expressions. + +Templates' drawback is often mentioned to be their dynamic nature which makes +static inferences hard or impossible. However, by again rewriting in clever +ways and making the code visible in a standard way, existing analysis code +can operate upon it. + +For the purposes of this discussion, templates can be broken down into three +distinct problems: + +A Finding what nodes to invoke upon. This is the expression found on +xsl:apply-templates/@select, in the case of template rules + +B Concluding what template to invoke. This is the analyzis and evaluation of +patterns, as found on xsl:template/@match, in the case of templates rules. +This is seen as a critical, as for instance Michael Kay emphasizes in Saxon: +Anatomy of an XSLT processor [LINK +http://www.ibm.com/developerworks/library/x-xslt2/] + +C Invoking the template for the given context node + +For these three steps, the two first are specific to template rules, while the +latter, invoking templates, can be seen to be treated identically regardless +of kind: template rules as well as named templates. + +With this perspective as background, lets try to write it into XQuery +primitives. + +First, all templates regardless of kind are instanciated by name. In the case +of templates rules, a synthetic name is given. They are invoked by an XPath +function named call-template() that as first argument takes the name of the +template, and also handles template parameters. This "template callsite" +which is separated from what it is invoked with and whether it is invoked, +knows its target template statically, and hence can be subject to inlining, +and usual functional analysis. + +Focus and concatenation of output handled. +One should consider whether templates couldn't be considered as functions, +with specialized arguments in the case of tunnel parameters. +Knowing what templates will be invoked could be used to conclude +node sorting is not necessary. +Mention how we do builtin templates + +Attribute Value Templates +------------------------- +XSL-T make extensive use of Attribute Value Templates(AVTs), which are handled +by turning the grammar piece in XQuery that is closest, into an expression. +Simply, ExprSingle[32] is extended with the branch: + +AVT LPAREN AttrValueContent RPAREN + +where AVT is a synthetic token XSLTokenizer generates. This means that the +code handling AVTs in XQuery's direct attribute constructors handles AVTs as +generic expressions. AttrValueContent creates a call to the concat() +function, over the operands. + +Deal with fn:current by using let $current := . return instruction. + +Another thing related to order and parsing is that XSL-T has more freedom wrt. +to where variables are in scope. For instance, a variable declaration appearing +after a user function declaration is in scope for the function in XSL-T, but +that's not the case in XQuery. This means that delayed variable resolution must +be added, something which wasn't, and cannot be active, for the XQuery code. +See 9.7 Scope of Variables. + +The parser generates for the builtin template rules: + + declare template matches (text() | @*) mode #all + { + text{.} + }; + + * +By having templates invocations essentially expressed as a callsite, or +branching, allows control flow analysis in a traditional manner, and hence the +possiblity to conclude what templates that are possibly invoked in various +contexts (or not invoked). One good example where this could improve template +matching is patterns containg predicates: let's say a template matches text +nodes with a predicate, but , doh I'm wrong. + +The problem with expressing template invocation with if expressions, is finding +ambiguous matches. + +Although normalizing down to a small set of primitives has its advantages, one +problem is with doing it too early. When doing it directly when tokenization, +the higher-level perspective is lost and therefore must be restored +again(example?). For instance, if an error is reported in a primitive, it must +not appear as originating from that primitive. It's not contstrained to error +reporting(example?). However, this is a general problem when compilers shift +between different representations. + +One effect this parsing approach has, is that the stylesheet cannot be used as +an input document(e.g, what document("") would evaluate to); in that case it +has to be parsed again. I think this is for the better; in the case that the +stylsheet has this dual role, it means representations are used which are +designed specifically for these respective roles. Although doing a dual parsing +step is costly, it's somewhat relieved by that the input is typically cached at +the byte level(file system and higher layers such as web/application caches) in +the case of traditional file loading. + +Another problem is that the grammar is used to solve implementation details, +and this might show as part of when the parser do error reporting. + +If one decide to not send XSL-T through the XQuery parser, it can be an +advantage to have as little business logic as possible in the XQuery parser +such that it can be reused. + +Some parts of XSL-T's syntax doesn't translate well to XQUery syntax. Some +parts doesn't follow structure very strongly, surely not the structures that +map well to XQuery's syntax. These are xml:base, @version and other attributes +that can appear on any element. Their information needs to be preserved and +need to affect the output code, but these cannot be done in a way which fits +naturally with the XQuery syntax, and hence leads to workarounds. Have whole +section on how we do @version and @xml:base. Another problem is namespace +declarations on the top document element. + +What largely makes me believe this technique fails is that the large and most +important parts, templates, instructions, maps well to XQuery, but the small +but yet not ignorable details like @version and @xml:base does not, to the +degree that the approach at large fails. + +fn:document() +------------------------ +See class documentation for DocumentFN. Document what optimizations one typically +wants to implement(const-fold on card 1, constant propagate). + + +In other words, it's reasonable to believe that it's possible to extend the +XQuery grammar such that it functionality wise is able to do the same as XSL-T, +but this doesn't equal that it is a good way to reach every gritty corner of +the XSL-T specification. + +Patterns +-------------------- +The up-side-down turning, discuss id/key(). + +Declarations +--------------------- +xsl:function: the 'declare function' declaration. TODO override + +XSL-T's error codes goes against good refactoring. Its codes are +specific on each usage, compared to for instance XPTY0004. + +Optimizations: string-join()/value-of + + <xsl:template match="document-node"> + <xsl:apply-templates select="child::element(doc)"/> + </xsl:template> + + <xsl:template match="child-or-top::element(doc)"/> + + => + + document-node()/child::element(doc) map apply-template + + matches child-or-top::element(doc) + + => + + N/root(.)//(EE) + + N == document-node() + EE == child::element(doc) + + => + + document-node()/root(.)/descendant-or-self::node()/child::element(doc) + + Optimize out already in createCopyOf() + +Bugs: + - DynamicContextStore and CurrentItemStore needs to implement + evaluateToReceiver(). + - Don't we have a parsing bug in each place where we call insideSequenceConstructor(), and don't + wrap the result in parantheses? E.g, a whitespace node followed by an instruction will lead to parse + error if the parent is for instance xsl:when. + +In patterns we find: + - Function :id() + - Function :key() + - AxisStep + - GenericPredicate. Also used for paths. + - (CombineNodes) + - empty sequence; attribute::foo()/child::asd + + +Test case, tokenizer asserts(fixed in 2a0e83b): + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> + + + <xsl:template match="/"> + <xsl:call-template name="TestFunction"/> + </xsl:template> + + + <xsl:template name="TestFunction"> + + + <xsl:call-template name="GetElem"> + <xsl:with-param name="element-set"select="$super/*"/> + </xsl:call-template> + + </xsl:template> + + <xsl:template name="GetElem"> + <xsl:param name="element-set"/> + <xsl:copy-of select="$element-set[@name]"/> + </xsl:template> + + </xsl:stylesheet> + +Typing code: + + <xsl:stylesheet xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> + + <xsl:template match="/"> + + <xsl:call-template name="templateName"> + <xsl:with-param name="a" select="2" /> + </xsl:call-template> + + </xsl:template> + + <xsl:template name="templateName"> + <xsl:param name="a" as="xs:integer"/> + <xsl:sequence select="$a"/> + </xsl:template> + + </xsl:stylesheet> + +Compat mode in attribute sets: + <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> + + <xsl:attribute-set name="attrSet" version="1.0"> + <xsl:attribute name="attributeName" select="1 + 'type error without compat mode'"/> + </xsl:attribute-set> + + <xsl:template match="/"> + <out xsl:use-attribute-sets="attrSet"/> + </xsl:template> + + </xsl:stylesheet> + +Space in mode: + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> + + <xsl:template match="/"> + <xsl:apply-templates mode=" #default"/> + </xsl:template> + + </xsl:stylesheet> + + +Type error in global template: + + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> + + <xsl:variable name="wa" as="item()"/><!-- HERE, incorrect cardinality. --> + + <xsl:template name="templateName"/> + + </xsl:stylesheet> + + +Variables are not in scope before its siblings: + + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> + + <xsl:template name="templateName"> + <xsl:sequence select="$var"/> + <xsl:variable name="var" select="1"/> + </xsl:template> + + </xsl:stylesheet> + +Crashes: + + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" + xmlns:xs="http://www.w3.org/2001/XMLSchema" + xmlns:local="http://example.com/" + version="2.0"> + + <xsl:variable name="var" as="xs:boolean"> + <xsl:value-of select="local:doesNotExist()"/> + </xsl:variable> + + </xsl:stylesheet> + + + +Whitespace handling, the important part is WS after xsl:template: + + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> + + <xsl:template match="/" xml:space="preserve"><MATCH/></xsl:template> + + </xsl:stylesheet> + + + +Whitespace handling, preserve, but not inside xsl:apply-templates: + + <xsl:stylesheet xmlns:xsl="http://www.w2.org/1999/XSL/Transform" version="2.0"> + + <xsl:template match="/" xml:space="preserve">MATCH<xsl:apply-templates> + + </xsl:apply-templates></xsl:template> + + </xsl:stylesheet> + +Have top-level xml:space, ensure whitespace as child of xsl:stylesheet is ignored: + + <xsl:stylesheet xml:space="preserve" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> + + <xsl:template match="/">MATCH<xsl:apply-templates> + + </xsl:apply-templates> + </xsl:template> + + </xsl:stylesheet> + +Compat mode, Saxon & QtXmlPatterns fails: + + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> + + <xsl:template match="/"> + <xsl:sequence version="1.0" select="string-join(current-dateTime(), 'separator')"/> + </xsl:template> + + </xsl:stylesheet> + +Compat mode, this is not in the suite: + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> + + <xsl:template match="/"> + <xsl:sequence version="1.0" select="subsequence((1, 2), '2')"/> + </xsl:template> + + </xsl:stylesheet> + +Crashes: + + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> + + <xsl:template match="doc"/> + + <xsl:apply-templates select="item" mode="crazy" /> + + + </xsl:stylesheet> + +Incorrectly yields compile error, XPST0003: + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> + + <xsl:template match=""/> + + <xsl:apply-templates select="item" mode="crazy" /> + + </xsl:stylesheet> + +Have a basic simplified stylesheet module: + + <output xsl:version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> + <xsl:value-of select="/"/> + </output> + +Have no @version: + <output xsl:version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> + <xsl:value-of select="/"/> + </output> + + +Is valid: + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0" > + + <xsl:template match="/"> + <xsl:perform-sort select="."> + <xsl:sort select="*"/> + <xsl:variable name="abc" select="b"/> + </xsl:perform-sort> + </xsl:template> + + </xsl:stylesheet> + + +Is valid: + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0" > + + <xsl:template match="/"> + <xsl:perform-sort select="."> + <xsl:sort select="*"/> + TEXT + </xsl:perform-sort> + </xsl:template> + + </xsl:stylesheet> + +XTSE0020: + <literalResultElement xsl:validation="disallowedValue"/> + +XTSE0020: + <xsl:element name="localName" validation="disallowedValue"/> + +XTSE0805: + <e xsl:disallowedAttribute=""/> + +not XPST0003, not in test suite: + + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> + <xsl:template match="/"> + <xsl:variable name="s" as="element()*"/> + </xsl:template> + </xsl:stylesheet> + +Parsing of many exprs in xsl:value-of(with separator): + + <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> + + <xsl:template match="/"> + <xsl:value-of separator="SEP"> + <xsl:sequence select="1"/> + <xsl:sequence select="2"/> + </xsl:value-of> + </xsl:template> + + </xsl:stylesheet> + + +Parsing of many exprs in xsl:value-of(without separator): + + <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> + + <xsl:template match="/"> + <xsl:value-of> + <xsl:sequence select="1"/> + <xsl:sequence select="2"/> + </xsl:value-of> + </xsl:template> + + </xsl:stylesheet> + + +Check type of empty variables: + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" + xmlns:xs="http://www.w3.org/2001/XMLSchema" + version="2.0"> + + <xsl:template match="/"> + <xsl:variable name="empty"/> + + <xsl:sequence select="'instance of xs:string:', $empty instance of xs:string, '(should be true)', + 'instance of document-node():', $empty instance of document-node(), '(should be false)', + 'value is:', $empty, + 'END'"/> + + </xsl:template> + + </xsl:stylesheet> + + +Crashes: + <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> + + <xsl:template match="/"> + <e xmlns="ABC"/> + </xsl:template> + + </xsl:stylesheet> + + +invalid standard attributes on a simplified stylesheet module. + + + + + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" + xmlns:xs="http://www.w3.org/2001/XMLSchema" + version="2.0"> + + <!-- Type error: applying templates to a variable of type string --> + <?spec xslt#applying-templates?> + <?error XTTE0520?> + + <xsl:template match="/"> + <xsl:variable name="empty"/> + + <xsl:sequence select="'instance of xs:string:', $empty instance of xs:string, 'instance of document-node():', $empty instance of document-node()"/> + + </xsl:template> + + </xsl:stylesheet> + + + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> + + <xsl:template match="/"> + <output> + <xsl:sequence select="string-length(doesNotMatch)"/> + </output> + </xsl:template> + + </xsl:stylesheet> + +Asserts(not wellformed): + + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> + <xsl:template match="/"> + <output> + </outpu> + </xsl:template> + + </xsl:stylesheet> + + +From within a function, use the focus /through/ a variable reference: + + <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:local="http://www.w3.org/2005/xquery-local-functions"> + + <xsl:variable name="var" select="node()"/> + + <xsl:function name="local:function"> + <xsl:sequence select="$var"/> + </xsl:function> + + <xsl:template match="/"> + <xsl:sequence select="local:function()"/> + </xsl:template> + + </xsl:stylesheet> + + + +Loops infinitely: + + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0"> + + <xsl:template match="/" version="1.0"> + <xsl:namespace name="{doc/item}" select="'http://www.example.com'" version="1.0"/> + </xsl:template> + + </xsl:stylesheet> + + +Gives crash in coloring code: + Stylesheet: + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> + + <xsl:template match="/"> + </xsl:template> + + </xsl:stylesheet> + + Focus: + <a><b/><</a> + + +Should evaluate to true: + + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema" version="2.0"> + + <xsl:template match="/"> + <xsl:call-template name="yo"> + <xsl:with-param name="arg" as="xs:integer"> + <xsl:sequence select="xs:untypedAtomic('1')"/> + </xsl:with-param> + </xsl:call-template> + </xsl:template> + + <xsl:template name="yo"> + <xsl:param name="arg"/> + <xsl:sequence select="$arg instance of xs:integer"/> + </xsl:template> + + </xsl:stylesheet> + +Crashes, should be XTTE0570: + <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" + xmlns:xs="http://www.w3.org/2001/XMLSchema" + exclude-result-prefixes="xs" version="2.0"> + + <xsl:template match="/"> + <xsl:apply-templates> + <xsl:with-param name="second_seq" as="xs:string"> + </xsl:with-param> + + </xsl:apply-templates> + </xsl:template> + + <xsl:template match="empty"> + <xsl:param name="second_seq">def</xsl:param> + <xsl:sequence select="$second_seq instance of xs:string"/> + </xsl:template> + + </xsl:stylesheet> + +* Parse error: + + <xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> + + <xsl:template match="/"> + <xsl:copy> + <xsl:sequence select="1"/> + <xsl:sequence select="2"/> + </xsl:copy> + </xsl:template> + + </xsl:stylesheet> + +* Write tests with xsl:with-param whose body is empty. That's effectively an + empty sequence(?) which needs to be handled properly, and (dynamically) type + checked correctly. + +-------------------------------------------------------------------------- + + + + + +-------------------------------------------------------------------------- + + + + ------------------------------------------------------------- + /a/b + + => + + b[parent::a[parent::document()]] + + but we currently have: + + (b[parent::a])[parent::document()] + + ------------------------------------------------------------- + a/b + + => + + b[parent::a] + + ------------------------------------------------------------- + a/b/c + + => + + c[parent::b[parent::a]] + + ------------------------------------------------------------- + a/b/c/d + + => + + d[parent::c[parent::b[parent::a]]] + + + ------------------------------------------------------------- + /a/b/c/d + + => + + d[parent::c[parent::b[parent::a[parent::document()]]]] + + This is handled specially; see | SLASH RelativePathPattern + + + b/c rewrites to: + TruthPredicate + AxisStep self::element(c) + AxisStep parent::element(b) + + For a/b/c we get: + + TruthPredicate + TruthPredicate + AxisStep self::element(c) + AxisStep parent::element(b) + AxisStep parent::element(a) + + But we want: + + TruthPredicate + AxisStep child-or-top::element(c) + TruthPredicate + AxisStep parent::element(b) + AxisStep parent::element(a) + + For a/b/c/d we get: + + TruthPredicate + TruthPredicate + TruthPredicate + AxisStep self::element(d) + AxisStep parent::element(c) + AxisStep parent::element(b) + AxisStep parent::element(a) + + For a/b/c/d we want: + + TruthPredicate + AxisStep self::element(d) + TruthPredicate + AxisStep parent::element(c) + TruthPredicate + AxisStep parent::element(b) + AxisStep parent::element(a) + + + For /a/b we get: + + TruthPredicate + TruthPredicate: + AxisStep self::element(b) + AxisStep parent::element(a) + AxisStep parent::document() + + but we want: + + TruthPredicate + AxisStep self::element(b) + TruthPredicate: // PREDICATE + AxisStep parent::element(a) + AxisStep parent::document() // PREDICATE + + -------------------------------------------------------------- + For a/b/c we get: + TruthPredicate + AxisStep self::element(c) + TruthPredicate + parent::element(b) + parent::element(a) + |