This document is intended to be used as a guideline for the testing activities related to the CSS Shapes Level 1 spec [[!css-shapes-1]]. Its main goal is to provide an overview of the general testing areas, possible caveats and testing aspects not immediately apparent from the spec. Also, it provides a means of tracking the progress of the CSS Shapes spec testing.

This document is not meant to replace the spec in determining the normative and non-normative assertions to be tested, but rather complement it.

Goals

To ensure a comprehensive test suite with useful, high quality tests, a number of goals are proposed. They range from process goals (how to conduct testing) to implementation goals (how to write good tests).

Enabling easy test contribution

An important vector in successfully testing CSS Shapes is to enable easy test contributions, both from W3C partners and from non-W3C members that wish to contribute. This is achieved by clearly marking and explaining the areas that need testing, linking to existing tests, and general testing progress.

Providing guidance on testing

In order to increase the quality of the test contributions, this document offers a set of guidelines for conducting testing (see ) and a testing progress tracker to increase the surface coverage of tests (see ).

Creating automation-friendly tests

In terms of actual tests produced for the CSS Shapes specification, the main goal is to ensure that most tests are automatable (i.e. they're either reftests or use testharness.js). Even where manual tests are absolutely necessary they should be written so that they can be easily automated – as there are ongoing efforts to make WebDriver [[webdriver]] automated tests a first class citizen in W3C testing. This means that even if a manual test requires user interaction, the validation or PASS/FAIL conditions should still be clear enough as to allow automatic validation if said interaction is later automated.

Approach

Since the CSS Shapes Level 1 spec introduces only three new CSS properties, the approach is to deep dive into every aspect of the spec as much as possible. Tests will be created for the testing areas listed in .

Testing areas

Explicit testing areas

These are testing areas normatively defined by the spec. They cover things explicitly or implicitly defined in the CSS Shapes spec. Please note that while detailed, this list is not necessarily exhaustive and normative behaviors may not be contained in it. When in doubt, consult the CSS Shapes spec or ask a question on the mailing list.

Proper parsing of the CSS properties and values according to the spec

Note: For all of the tests below, where length parameters are tested, the supported <length> units defined in the CSS Values & Units [[!CSS3VAL]] specification, but not all permutations will not be tested. Instead, a sampling of these units will be used across the parsing and layout tests.

  • shape-outside
    • none
    • <basic-shape>
      • inset()
        • 0-4 arguments
        • length units
        • percentages
        • positive/negative lengths
        • decimal/non-decimal lengths
        • calc() lengths
        • no unit or %
        • commas / no commas
        • invalid arg values
        • round keyword
          • <border-radius>
            • 0-8 arguments
            • position of '/'
            • length units
            • percentages
            • positive/negative lengths
            • decimal/non-decimal lengths
            • calc() lengths
            • no unit or %
            • commas / no commas
            • invalid values
      • circle()
        • <shape-radius>
          • 0-1 args (valid), 2 args (invalid)
          • length units
          • percentages
          • positive/negative lengths
          • decimal/non-decimal lengths
          • calc() lengths
          • no unit or %
          • commas / no commas
          • invalid arg values
          • closest-side, farthest-side keywords
        • at keyword
          • <position>
            • 0-4 arguments
            • length units
            • percentages
            • positive/negative lengths
            • decimal/non-decimal lengths
            • calc() lengths
            • no unit or %
            • commas / no commas
            • invalid values
            • top, left, bottom, right keywords
      • ellipse()
        • <shape-radius>
          • 0-2 args (valid), 3 args (invalid)
          • Plus all of the same shape-radius tests listed above for circle()
        • Plus all of the at <position> tests listed above for circle()
      • polygon()
      • <shape-arg>'s
        • 1-6 vertices (valid), 0 vertices (invalid)
        • length units
        • percentages
        • positive/negative lengths
        • decimal/non-decimal lengths
        • calc() lengths
        • no unit or %
        • commas / no commas
        • invalid arg values
      • <fill-rule>
        • not specified (default: nonzero)
        • nonzero
        • evenodd
        • invalid values
    • <box>
      • margin-box
      • border-box
      • padding-box
      • content-box
    • <basic-shape> + <box>
      • A sampling of the tests above combined together to test basic-shape with shape-box
    • <image>
  • shape-margin
    • length units
    • percentages
    • positive/negative lengths
    • decimal/non-decimal lengths
    • calc() lengths
    • no unit or %
    • invalid values
  • shape-image-threshold
    • decimal - 0-5 places
    • no decimal
    • percentage (invalid)
    • positive/negative values
    • decimal/non-decimal lengths
    • calc() lengths
    • + / - signs
    • invalid values

Proper serialization of the CSS properties and values according to the spec

  • <position> values in circle and ellipse serialize to 2- and 4-value forms
    • top, left, bottom, right serialize to percentages
    • omitting radii omits radii from serialization
  • inset serialized shape-args omit args when possible
  • <round> values in inset serialize to as few as possible
  • unspecified <fill-rule> serializes nonzero
  • 0% is preferred over zero length
  • calc() is avoided

Proper computed values of the CSS properties and values according to the spec

  • lengths compute to px
  • percentages keep %
  • calc() preserved

Proper rendering & layout of text around shapes according to the spec

  • Basic Shapes
    • inset
      • square corners
      • rounded corners
      • rounded corners to make circles & ellipses
    • circle, ellipse
      • explicit/implicit radii
      • explicit/implicit position
      • closest-side / farthest-side
    • polygon
      • relative/absolute units
      • fill-rule: evenodd/nonzero
    • float left / right
    • shape-margin
    • shape-box
      • unspecified
      • specified:
        • margin-box
        • border-box
        • padding-box
        • content-box
  • Shapes from Box Values
    • margin-box, border-box, padding-box, content-box
    • all, top, left, bottom, right, top-left, top-bottom, top-right, left-right, left-bottom, right-bottom
    • float left / right
    • border-radius
  • Shapes from Images
    • url
      • png, svg, gif, jpeg
      • transparency/no transparency
    • image-list
      • png, svg, gif, jpeg
      • transparency/no transparency
    • gradient
      • linear
      • radial
      • repeating
    • float left / right
    • shape-margin
    • shape-image-threshold
  • Float Tests
    • Float stacking
    • Line boxes affected by both float right and float left shapes at the same time
    • Empty float areas
    • Shapes that extend past the margin box edge

Implicit testing areas

These are testing areas either normatively defined in other specs that are explicitly referred to in the CSS Shapes spec. Additionally, the CSS Shapes spec is explicitly referred to by other specs. Please note that while detailed, this list is not necessarily exhaustive and normative behaviors may not be contained in it. When in doubt, consult the CSS Regions spec or ask a question on the mailing list.

Below is the list of implicit testing areas:

Interactions with other CSS features & specifications

When the CSS Shapes spec has a full suite of tests covering the behaviors defined in the spec, additional tests will be needed to assure that the implementation works properly with other CSS features defined in other specifications.

Below is a list of features that should work properly with CSS Shapes:

People and responsibilities

Below is a list of people you should reach out to if you have any questions related to this document or testing CSS Shapes in general:

Test progress tracking

Currently test progress tracking is done via gitHub milestones and issues.