[css-images-4] image-rendering: pixelated, scaling up/down?

[Simon Sapin]

Hi,

The spec for the 'pixelated' value of the 'image-rendering' property has
different text for "When scaling the image up" and "When scaling down".
How is that defined?

First, should this only apply to raster images, like 'image-resolution'?

Assuming that, I think the distinction of scaling up or down should be
based on the ratio of image pixels per device pixels, regardless of
anything measured in CSS units.

Some cases are more tricky:

* What if one dimension is scaled up and the other down? For example a
100x100 PNG with `width: 200px; height: 50px`.

* What if the overall transform is not axis-aligned? For example a
rotation angle that is not a multiple of 90 degrees.


Perhaps "When scaling the image up in at least one dimension" is enough.

[Daniel Holbert]

Post by Simon Sapin
The spec for the 'pixelated' value of the 'image-rendering' property has
different text for "When scaling the image up" and "When scaling down".
How is that defined?

[...]

Post by Simon Sapin
* What if one dimension is scaled up and the other down? For example a
100x100 PNG with `width: 200px; height: 50px`.

[...]

Post by Simon Sapin
Perhaps "When scaling the image up in at least one dimension" is enough.

I'm curious about this as well. (Looks like this spec text still makes a
binary distinction between "scaling up" vs "scaling down", which doesn't
actually cover the space of possibilities, as Simon says above.)

Could this be clarified in the spec?

For reference, it looks like Blink's implementation does what Simon
suggested -- it pixelates when *at least one* dimension is being scaled up:
http://src.chromium.org/viewvc/blink/trunk/Source/core/rendering/ImageQualityController.cpp?r1=176839&r2=176957&pathrev=176957
http://src.chromium.org/viewvc/blink/trunk/Source/core/rendering/RenderHTMLCanvas.cpp?r1=176839&r2=176957&pathrev=176957

~Daniel

[Daniel Holbert]

Post by Daniel Holbert

Post by Simon Sapin
The spec for the 'pixelated' value of the 'image-rendering' property has
different text for "When scaling the image up" and "When scaling down".
How is that defined?

[...]

Post by Simon Sapin
* What if one dimension is scaled up and the other down? For example a
100x100 PNG with `width: 200px; height: 50px`.

[...]

Post by Simon Sapin
Perhaps "When scaling the image up in at least one dimension" is enough.

I'm curious about this as well. (Looks like this spec text still makes a
binary distinction between "scaling up" vs "scaling down", which doesn't
actually cover the space of possibilities, as Simon says above.)
Could this be clarified in the spec?
For reference, it looks like Blink's implementation does what Simon
http://src.chromium.org/viewvc/blink/trunk/Source/core/rendering/ImageQualityController.cpp?r1=176839&r2=176957&pathrev=176957
http://src.chromium.org/viewvc/blink/trunk/Source/core/rendering/RenderHTMLCanvas.cpp?r1=176839&r2=176957&pathrev=176957

Spec quote/link, for reference:
# When scaling the image up, [...]
# When scaling down, [...]
http://dev.w3.org/csswg/css-images/#valuedef-pixelated

[I'm also updating the email-subject to drop the "-4", to match the
requested "Feedback" format from the current ED.]

[Tab Atkins Jr.]

Post by Daniel Holbert

Post by Daniel Holbert

Post by Simon Sapin
The spec for the 'pixelated' value of the 'image-rendering' property has
different text for "When scaling the image up" and "When scaling down".
How is that defined?

[...]

Post by Simon Sapin
* What if one dimension is scaled up and the other down? For example a
100x100 PNG with `width: 200px; height: 50px`.

[...]

Post by Simon Sapin
Perhaps "When scaling the image up in at least one dimension" is enough.

I'm curious about this as well. (Looks like this spec text still makes a
binary distinction between "scaling up" vs "scaling down", which doesn't
actually cover the space of possibilities, as Simon says above.)
Could this be clarified in the spec?
For reference, it looks like Blink's implementation does what Simon
http://src.chromium.org/viewvc/blink/trunk/Source/core/rendering/ImageQualityController.cpp?r1=176839&r2=176957&pathrev=176957
http://src.chromium.org/viewvc/blink/trunk/Source/core/rendering/RenderHTMLCanvas.cpp?r1=176839&r2=176957&pathrev=176957

# When scaling the image up, [...]
# When scaling down, [...]
http://dev.w3.org/csswg/css-images/#valuedef-pixelated

How... how did I miss this email.

Fixed.

~TJ

[Daniel Holbert]

Thanks! On the same topic -- is there documentation anywhere (or a
thread on this list) about why there's a required difference in behavior
between downscaling & upscaling, for this "pixelated" value?

From reading the Blink intent-to-implement thread[1], it sounds like

part of the reason may be that downscaling with nearest-neighbor doesn't
actually *look* "pixelated" at all. This make some sense, but I'm not
sure it makes sense to me that this reason (on its own) is enough to
jusify requiring additional logic for "are we upscaling or downscaling"
for every image-rendering-flavored operation that's influenced by this
particular style. (and I wouldn't expect authors to be too mystified if
"image-rendering:pixelated" produced gross, blocky behavior when
downscaling.)

Were there other reasons as well, or is that mostly it?

Thanks,
~Daniel

[1] In particular:
https://groups.google.com/a/chromium.org/d/msg/blink-dev/Q8N6FoeoPXI/bq5U8544kE8J
https://groups.google.com/a/chromium.org/d/msg/blink-dev/Q8N6FoeoPXI/NPjrZxTLKAgJ
...and surrounding messages.

[Tab Atkins Jr.]

Post by Daniel Holbert
Thanks! On the same topic -- is there documentation anywhere (or a
thread on this list) about why there's a required difference in behavior
between downscaling & upscaling, for this "pixelated" value?
From reading the Blink intent-to-implement thread[1], it sounds like
part of the reason may be that downscaling with nearest-neighbor doesn't
actually *look* "pixelated" at all. This make some sense, but I'm not
sure it makes sense to me that this reason (on its own) is enough to
jusify requiring additional logic for "are we upscaling or downscaling"
for every image-rendering-flavored operation that's influenced by this
particular style. (and I wouldn't expect authors to be too mystified if
"image-rendering:pixelated" produced gross, blocky behavior when
downscaling.)
Were there other reasons as well, or is that mostly it?

Nope, that's it. The point of the keyword is to capture an intent,
and downscaling with nearest-neighbor doesn't actually do that, so I
had it instead do a higher-quality downscale.

That said, I'm not too concerned about this. If it's a bother to
track upscaling vs downscaling, I'm fine with just making it
consistent.

~TJ

[Daniel Holbert]

Post by Tab Atkins Jr.
Nope, that's it. The point of the keyword is to capture an intent,
and downscaling with nearest-neighbor doesn't actually do that, so I
had it instead do a higher-quality downscale.
That said, I'm not too concerned about this. If it's a bother to
track upscaling vs downscaling, I'm fine with just making it
consistent.

Thanks!

Since you offered... I have to say, it is somewhat of a bother. :) in
Gecko, in the function[1] that we converts "image-rendering" into an
enum that represents a specific scaling algorithm, we don't have access
to the intrinsic size or render size, so we can't correctly resolve
"pixelated" there. (Many callers of that function don't necessarily
have access to these sizes, either.)

To support the distinct upscaling/downscaling behavior, we'd probably
have to add a new dummy "pixelated" enum to our list of scaling
algorithms, and then add special cases to handle it in the code that
deals with this enum type. This is all definitely possible, but simply
treating "pixelated" as "nearest-neighbor" would be much simpler.

Also, the initial Blink implementation of "pixelated" had trouble with
tracking upscaling vs. downscaling, too -- it didn't pixelate when it
should've, in a case with HiDPI monitors[1]. So, they needed special
cases as well, and those special-cases were non-trivial to get right.

So unless there's a strong reason for the upscaling/downscaling
distinction, I'd support changing the spec to make it consistent.

~Daniel

[1] https://code.google.com/p/chromium/issues/detail?id=391351

[Tab Atkins Jr.]

Post by Daniel Holbert

Post by Tab Atkins Jr.
Nope, that's it. The point of the keyword is to capture an intent,
and downscaling with nearest-neighbor doesn't actually do that, so I
had it instead do a higher-quality downscale.
That said, I'm not too concerned about this. If it's a bother to
track upscaling vs downscaling, I'm fine with just making it
consistent.

Thanks!
Since you offered... I have to say, it is somewhat of a bother. :) in
Gecko, in the function[1] that we converts "image-rendering" into an
enum that represents a specific scaling algorithm, we don't have access
to the intrinsic size or render size, so we can't correctly resolve
"pixelated" there. (Many callers of that function don't necessarily
have access to these sizes, either.)
To support the distinct upscaling/downscaling behavior, we'd probably
have to add a new dummy "pixelated" enum to our list of scaling
algorithms, and then add special cases to handle it in the code that
deals with this enum type. This is all definitely possible, but simply
treating "pixelated" as "nearest-neighbor" would be much simpler.
Also, the initial Blink implementation of "pixelated" had trouble with
tracking upscaling vs. downscaling, too -- it didn't pixelate when it
should've, in a case with HiDPI monitors[1]. So, they needed special
cases as well, and those special-cases were non-trivial to get right.
So unless there's a strong reason for the upscaling/downscaling
distinction, I'd support changing the spec to make it consistent.

Done.

~TJ

[Daniel Holbert]

Post by Daniel Holbert
Since you offered... I have to say, it is somewhat of a bother. :) in
Gecko, in the function[1] that we converts "image-rendering" into an
enum that represents a specific scaling algorithm, we don't have access
to the intrinsic size or render size, so we can't correctly resolve
"pixelated" there. (Many callers of that function don't necessarily
have access to these sizes, either.)

(Sorry, I meant to link [1] to this source reference:
http://mxr.mozilla.org/mozilla-central/source/layout/base/nsLayoutUtils.cpp#4924
)

[Lea Verou]

Post by Simon Sapin
Hi,
The spec for the 'pixelated' value of the 'image-rendering' property has
different text for "When scaling the image up" and "When scaling down".
How is that defined?
First, should this only apply to raster images, like 'image-resolution'?
Assuming that, I think the distinction of scaling up or down should be
based on the ratio of image pixels per device pixels, regardless of
anything measured in CSS units.
* What if one dimension is scaled up and the other down? For example a
100x100 PNG with `width: 200px; height: 50px`.
* What if the overall transform is not axis-aligned? For example a
rotation angle that is not a multiple of 90 degrees.
Perhaps "When scaling the image up in at least one dimension" is enough.
--
Simon Sapin

Hi Simon,

It looks like there is no mention of "scaling up" or "scaling down" in the prose anymore. Instead the text now reads "The image must be scaled with the "nearest neighbor" or similar algorithm, to preserve a "pixelated" look as the image changes in size".

Do you feel your issue has been resolved?

Cheers,
Lea

Lea Verou ✿ http://lea.verou.me ✿ @leaverou

[Simon Sapin]

Post by Lea Verou
Hi Simon,
It looks like there is no mention of "scaling up" or "scaling down" in
the prose anymore. Instead the text now reads "The image must be scaled
with the "nearest neighbor" or similar algorithm, to preserve a
"pixelated" look as the image changes in size".
Do you feel your issue has been resolved?

Yes. Thank you.