Are you interested in doing dynamic analysis of JS code?

[Jason Orendorff]

We're considering building a JavaScript API for dynamic analysis of JS code.
Here's the sort of thing you could do with it:

- Gather code coverage information (useful for testing/release mgmt?)

- Trace all object mutation and method calls (useful for devtools?)

- Record/replay of JS execution (useful for devtools?)

- Implement taint analysis (useful for the security team or devtools?)

- Detect when a mathematical operation returns NaN (useful for game
developers?)

Note that the API would not directly offer all these features. Instead, it
would offer some powerful but mind-boggling way of instrumenting all JS
code. It would be up to you, the user, to configure the instrumentation, get
useful data out of it, and display or analyze it. There would be some
overhead
when you turn this on; we don't know how much yet.

We would present a detailed example of how to use the proposed API, but
we are
so early in the process that we're not even sure what it would look like.
There are several possibilities.

We need to know how to prioritize this work. We need to know what kind
of API
we should build. So we're looking for early adopters. If that's you, please
speak up and tell us how you'd like to instrument JS code.

[Frederik Braun]

Thanks for bringing this to dev-platform.

Dynamic analysis is something the security teams are particularly
interested in. Especially tainting user input is something we could make
use of across the project: Existing security efforts for Firefox OS,
Firefox Desktop, Firefox Mobile and our websites would all greatly
benefit from it, as it could help preventing Cross-Site Scripting and
other content injection attacks.

Some people may know the work Stefano Di Paola has done to develop his
DOM-XSS scanner "DOMinator". There's also been an attempt to develop it
in-tree within the security mentorship program, but the outcome wasn't
fit to be merged into moz-central (bug 811877).

A mozilla-owned API would help make all future endeavors last. I have
also been in contact with folks in academia and the industry who are
interested in both implementation and consumption of the API.

I will make sure their attention is directed to this threat to provide
additional feedback.

[Garrett Robinson]

Tainting could also be of use in a particular problem area for Content
Security Policy (CSP): allowing modifications to CSP-protected pages
caused by add-ons or bookmarklets. At the moment, such modifications
(e.g. an add-on injecting tags into a page) are indistinguishable from
malicious content injection attacks, and so are blocked. Unfortunately,
the end result for users is their addons and bookmarklets "break" on
these CSP-protected pages - a problem that will only worsen as CSP
adoption increases.

It is debatable whether such modifications should be allowed. According
to the priority of constituencies, the user/user agent takes precedence
over the the site, so add-ons and bookmarklets (typically interpreted as
acting with the knowledge and consent on the user) should be able to
override a CSP provided by the site. However, it is not clear that this
interpretation is always valid (consider malicious addons, crapware
addons, or default addons included with the user's knowledge by the OS
or manufacturer).

Either way, tainting support in the JS engine, and general
instrumentation/metadata for JS calls, would probably help in achieving
this goal (although we'd probably also have to add taint information to
DOM objects as well, so CSP knows when it should be bypassed).

Post by Frederik Braun
Thanks for bringing this to dev-platform.
Dynamic analysis is something the security teams are particularly
interested in. Especially tainting user input is something we could make
use of across the project: Existing security efforts for Firefox OS,
Firefox Desktop, Firefox Mobile and our websites would all greatly
benefit from it, as it could help preventing Cross-Site Scripting and
other content injection attacks.
Some people may know the work Stefano Di Paola has done to develop his
DOM-XSS scanner "DOMinator". There's also been an attempt to develop it
in-tree within the security mentorship program, but the outcome wasn't
fit to be merged into moz-central (bug 811877).
A mozilla-owned API would help make all future endeavors last. I have
also been in contact with folks in academia and the industry who are
interested in both implementation and consumption of the API.
I will make sure their attention is directed to this threat to provide
additional feedback.
_______________________________________________
dev-platform mailing list
https://lists.mozilla.org/listinfo/dev-platform

[Ted Mielczarek]

Post by Jason Orendorff
We're considering building a JavaScript API for dynamic analysis of JS code.
- Gather code coverage information (useful for testing/release mgmt?)

There have been several JS code coverage implementations that have
fallen by the wayside because of the difficulty of both making them work
across all our test suites and keeping them working. Something built on
a well-supported platform API would be a welcome change.

-Ted

[Fitzgerald, Nick]

Yes!!

Post by Jason Orendorff
We're considering building a JavaScript API for dynamic analysis of JS code.
- Gather code coverage information (useful for testing/release mgmt?)

Yes! We'd absolutely love to show code coverage in the debugger's source
editor! We've played with implementations based on
Debugger.prototype.onEnterFrame and hidden breakpoints, but it is pretty
slow and feels like a huge hack. Would this work likely yield improved
performance over that approach?

Post by Jason Orendorff
- Trace all object mutation and method calls (useful for devtools?)

Absolutely! A thousand times yes! We've made a prototype tracing
debugger (again, based on Debugger.prototype.onEnterFrame) but (again)
it is pretty slow so it isn't pref'd on by default (a few other reasons
as well; it needs polish).

We (devtools) have talked a lot about wanting to be the "best printf
debugger" to lower the barriers to entry for people who normally print
debug. Manually adding printfs is like a really crappy, temporary,
tracing debugger; I see huge opportunities for developer productivity here!

(Flip devtools.debugger.tracer to see the prototype:
Loading Image...)

Post by Jason Orendorff
- Record/replay of JS execution (useful for devtools?)

I'm drooling aaaahhhhhhhhhh

Post by Jason Orendorff
- Detect when a mathematical operation returns NaN (useful for game
developers?)

This would be a great fit for a console warning, IMO.

Post by Jason Orendorff
Note that the API would not directly offer all these features.
Instead, it
would offer some powerful but mind-boggling way of instrumenting all JS
code. It would be up to you, the user, to configure the
instrumentation, get
useful data out of it, and display or analyze it. There would be some
overhead
when you turn this on; we don't know how much yet.

Would the API be something like DTrace? Just want to figure out what
kind of thing we are talking about here.

Post by Jason Orendorff
We would present a detailed example of how to use the proposed API,
but we are
so early in the process that we're not even sure what it would look like.
There are several possibilities.
We need to know how to prioritize this work. We need to know what kind
of API
we should build. So we're looking for early adopters. If that's you, please
speak up and tell us how you'd like to instrument JS code.

/me raises hand

Mostly interested in tracing calls and mutations. Also code coverage,
but to a bit of a lesser extent.

Record/replay is such a holy grail (eclipsed only by "time traveling" /
reverse and replay interactive (as opposed to a static recording)
debugging with live, on-stack code editing) that I hesitate to even get
my hopes up...

Nick

[Mike de Boer]

Record/replay is such a holy grail (eclipsed only by "time traveling" / reverse and replay interactive (as opposed to a static recording) debugging with live, on-stack code editing) that I hesitate to even get my hopes up…

Oh, the joy I had while working with Microsoft Script Editor over 10 years ago with which I could step out, step out, step out… move the arrow in the gutter pointing at the current line upward… hit ‘Next’… magic!

If only I could relive those days with our devtools debugger! That’d be a dream come true!

MikeDreamy.

[Jason Orendorff]

Post by Fitzgerald, Nick
Yes! We'd absolutely love to show code coverage in the debugger's
source editor! We've played with implementations based on
Debugger.prototype.onEnterFrame and hidden breakpoints, but it is
pretty slow and feels like a huge hack. Would this work likely yield
improved performance over that approach?

If this pans out, I'd expect it to be faster, but how much depends on a
lot of variables. I wish I could be more specific.

Post by Fitzgerald, Nick

Post by Jason Orendorff
Note that the API would not directly offer all these features. Instead, it
would offer some powerful but mind-boggling way of instrumenting all JS
code. It would be up to you, the user, to configure the
instrumentation, get
useful data out of it, and display or analyze it. There would be some
overhead
when you turn this on; we don't know how much yet.

Would the API be something like DTrace? Just want to figure out what
kind of thing we are talking about here.

Very good question. I too am interested in figuring out what kind of
thing we're talking about.

One proposal is to build something like strace: it would be impossible
to modify the instrumented code or its execution, only observe.
User-specified data about JS execution would be logged, then delivered
asynchronously. (Don't read too much into this -- the implementation
would be completely unlike strace. Note too that records would *not* be
delivered synchronously and would not contain stacks, though you could
recover the stack from enter/leave records.)

An alternative involves letting you modify JS code just before it's
compiled (source-to-source transformation). This is more general (you
could modify the instrumented code arbitrarily, and react synchronously
as it executes) but maybe that's undesirable. It's not clear that
transformed source would interact nicely with other tools, like the
debugger. And a usable API for this is a tall order.

So. Tradeoffs.

Post by Fitzgerald, Nick
/me raises hand
Mostly interested in tracing calls and mutations. Also code coverage,
but to a bit of a lesser extent.

Great, we'll get in touch off-list!

Post by Fitzgerald, Nick
Record/replay is such a holy grail (eclipsed only by "time traveling"
/ reverse and replay interactive (as opposed to a static recording)
debugging with live, on-stack code editing) that I hesitate to even
get my hopes up...

Don't get your hopes up.

A dynamic analysis API would help with the "record" side of rr-style
record/replay **of JS code alone**. You'd draw a boundary around all JS
code and capture all input across that boundary. But replay requires
more work in the engine. A bigger problem is that in such a scheme,
replay only reproduces what happens inside the boundary, and the DOM, in
this scenario, is still on the outside. A real record/replay product
would have to support the DOM too. We're far from being able to do that.

The dynamic analysis API would only be one part of the puzzle. I'm sorry
for the misleading gloss.

-j

[Fitzgerald, Nick]

Post by Jason Orendorff
An alternative involves letting you modify JS code just before it's
compiled (source-to-source transformation). This is more general (you
could modify the instrumented code arbitrarily, and react
synchronously as it executes) but maybe that's undesirable. It's not
clear that transformed source would interact nicely with other tools,
like the debugger. And a usable API for this is a tall order.

We explored source level instrumentation using the proposed
onBeforeSourceCompiled hook
(https://bugzilla.mozilla.org/show_bug.cgi?id=884602), and that was a
speed improvement (although it required a reload to enable).
Unfortunately, priorities caught up with us and we haven't had a chance
to revisit.

It /should/ be fine with the debugger so long as you aren't doing too
invasive of changes (eg rearranging scopes) and you generate a source
map between the original source and the new source. Unfortunately, that
can also slow down the transformation quite a bit.

[Robert O'Callahan]

Post by Jason Orendorff
An alternative involves letting you modify JS code just before it's
compiled (source-to-source transformation). This is more general (you could
modify the instrumented code arbitrarily, and react synchronously as it
executes) but maybe that's undesirable. It's not clear that transformed
source would interact nicely with other tools, like the debugger. And a
usable API for this is a tall order.

Why is a usable S2S API difficult to produce?

A while ago I spent a few years doing dynamic analysis of Java code.
Although the VMs had a lot of tracing and logging hooks, bytecode
instrumentation was always more flexible and, done carefully, almost always
more performant. I had to write some libraries to make it easy but tool
builders enjoy writing and reusing those :-).

For JS of course we wouldn't want to expose our internal bytecode, hence
S2S.

Rob

[Katelyn Gadd]

Record/replay would be incredibly useful for game developers trying to
do automated testing or go back and analyze a rare failure case that
happens occasionally. I already do a bunch of this on my end by
recording API calls and results and such, having it at a lower JS
level would be incredibly useful.

Being able to detect operations that produced NaN is also handy for
that sort of debugging or even as a way to just assert that it never
happens in debug builds of your game.

The other features listed sound useful as well but I don't know if
they'd ever get used by game developers. I'm not sure I'd use them for
JSIL either.

Maybe you could use mutation/method call tracing to do a
profile-guided-optimization equivalent for JS where you record
argument types and clone functions per-callsite to ensure everything
is monomorphic? I could see that being a big performance boon,
especially if you can do it statically and automatically by using
instrumentation.

Post by Robert O'Callahan

Post by Jason Orendorff
An alternative involves letting you modify JS code just before it's
compiled (source-to-source transformation). This is more general (you could
modify the instrumented code arbitrarily, and react synchronously as it
executes) but maybe that's undesirable. It's not clear that transformed
source would interact nicely with other tools, like the debugger. And a
usable API for this is a tall order.

Why is a usable S2S API difficult to produce?
A while ago I spent a few years doing dynamic analysis of Java code.
Although the VMs had a lot of tracing and logging hooks, bytecode
instrumentation was always more flexible and, done carefully, almost always
more performant. I had to write some libraries to make it easy but tool
builders enjoy writing and reusing those :-).
For JS of course we wouldn't want to expose our internal bytecode, hence
S2S.
Rob
--
Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa
stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr,
'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp
waanndt wyeonut thoo mken.o w
_______________________________________________
dev-platform mailing list
https://lists.mozilla.org/listinfo/dev-platform

[Jason Orendorff]

Post by Katelyn Gadd
Maybe you could use mutation/method call tracing to do a
profile-guided-optimization equivalent for JS where you record
argument types and clone functions per-callsite to ensure everything
is monomorphic?

Yes, I think this is something that could be done with it.

But! Note that we're separately working on exposing JIT performance
faults in the profiler/devtools---directly reporting the kind of
problems you're thinking about painstakingly inferring. \o/

I don't know how far along it is, and it's maybe too early to guess how
actionable the output will be, but it's a much more certain thing than
this dynamic analysis API. No meta bug, but several people are working
on aspects of it. Maybe Kannan can tell more. (cc-ing him)

-j

[Shu-yu Guo]

Hi Katelyn,

I'm going to refer to what you're talking about with cloning and monomorphism as the "context sensitivity problem". As far as I know, this is still an open problem (not in the fundamental sense, just that nobody has a good solution for it yet) in the JS JITs. The argument types (counting 'this' as an argument) is one part of the context; the other part being the types of its free variables. For example, the context sensitivity problem is part of the reason repeated calls to something like Array.prototype.map is slow, and why any user-written collections libraries with higher order functions will be slow.

I've experimented with manually hinting functions to be cloned at callsite. We do this currently in SpiderMonkey only for certain PJS functions, like mapPar. This is a pretty brittle optimization though, since it can be easily defeated by wrapping, e.g., function myMapPar(arr, fn) { arr.mapPar(fn); }. I'd love to solve this problem in general. Such a solution would require:

1) A notion of "context" that's efficient in both time and space (e.g. easy to compute and cheap to store).
2) A notion of "context" that's robust (e.g. not defeated by wrapping).
3) Ability to determine what functions benefit from being cloned per-context.

PGO sounds like a great way to address 3) that I haven't thought of before. I'd also love to hear ideas about 1) and 2).

Cheers,

----- Original Message -----
From: "Jason Orendorff" <jorendorff-***@public.gmane.org>
To: kg-***@public.gmane.org, "Robert O'Callahan" <robert-***@public.gmane.org>
Cc: "Kannan Vijayan" <kvijayan-***@public.gmane.org>, dev-platform-CzyLcWPZiU5YsZ3hbOqMTti2O/***@public.gmane.org, fitzgen-***@public.gmane.org
Sent: Friday, June 27, 2014 1:21:02 PM
Subject: Exposing JIT performance faults in profiler/devtools (was: Are you interested in doing dynamic analysis of JS code?)

Post by Katelyn Gadd
Maybe you could use mutation/method call tracing to do a
profile-guided-optimization equivalent for JS where you record
argument types and clone functions per-callsite to ensure everything
is monomorphic?

Yes, I think this is something that could be done with it.

But! Note that we're separately working on exposing JIT performance
faults in the profiler/devtools---directly reporting the kind of
problems you're thinking about painstakingly inferring. \o/

I don't know how far along it is, and it's maybe too early to guess how
actionable the output will be, but it's a much more certain thing than
this dynamic analysis API. No meta bug, but several people are working
on aspects of it. Maybe Kannan can tell more. (cc-ing him)

-j

[Jason Orendorff]

On Thu, Jun 26, 2014 at 8:06 AM, Jason Orendorff
An alternative involves letting you modify JS code just before
it's compiled (source-to-source transformation). This is more
general (you could modify the instrumented code arbitrarily, and
react synchronously as it executes) but maybe that's undesirable.
It's not clear that transformed source would interact nicely with
other tools, like the debugger. And a usable API for this is a
tall order.
Why is a usable S2S API difficult to produce?
A while ago I spent a few years doing dynamic analysis of Java code.
Although the VMs had a lot of tracing and logging hooks, bytecode
instrumentation was always more flexible and, done carefully, almost
always more performant. I had to write some libraries to make it easy
but tool builders enjoy writing and reusing those :-).

I meant "usable" in the Donald Norman sense. The underlying
source-transformation hook could be quite simple, and yet completely
beyond the people who actually have a use for it. How many potential
users are willing and ready to munge ASTs by hand?

Bug 884602 contains a patch for the hook alone. I'd love to
- land that patch (it needs tests)
- write an addon that makes it easy to play with instrumentation of
content JS
- post a video demoing it

But it's very hard to justify doing that instead of, say, ES6 classes.

-j

[k***@public.gmane.org]

I agree with Rob that S2S would give the most flexibility. Something similar to java.lang.instrumentation API would be immensely useful (http://docs.oracle.com/javase/7/docs/api/java/lang/instrument/Instrumentation.html). I am a maintainer and developer of Jalangi (https://github.com/SRA-SiliconValley/jalangi) and I believe a java.lang.instrumentation like API would help us to make Jalangi easily accessible to developers.

On Thu, Jun 26, 2014 at 8:06 AM, Jason Orendorff

Post by Jason Orendorff
An alternative involves letting you modify JS code just before it's
compiled (source-to-source transformation). This is more general (you could
modify the instrumented code arbitrarily, and react synchronously as it
executes) but maybe that's undesirable. It's not clear that transformed
source would interact nicely with other tools, like the debugger. And a
usable API for this is a tall order.

Why is a usable S2S API difficult to produce?
A while ago I spent a few years doing dynamic analysis of Java code.
Although the VMs had a lot of tracing and logging hooks, bytecode
instrumentation was always more flexible and, done carefully, almost always
more performant. I had to write some libraries to make it easy but tool
builders enjoy writing and reusing those :-).
For JS of course we wouldn't want to expose our internal bytecode, hence
S2S.
Rob
--
Jtehsauts tshaei dS,o n" Wohfy Mdaon yhoaus eanuttehrotraiitny eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o Whhei csha iids teoa
stiheer :p atroa lsyazye,d 'mYaonu,r "sGients uapr,e tfaokreg iyvoeunr,
'm aotr atnod sgaoy ,h o'mGee.t" uTph eann dt hwea lmka'n? gBoutt uIp
waanndt wyeonut thoo mken.o w

[Gregory Szorc]

Post by Jason Orendorff
We're considering building a JavaScript API for dynamic analysis of JS code.
- Gather code coverage information (useful for testing/release mgmt?)

As someone who develops JS for Firefox features, code coverage is near
the top of my list of wants.

I want it as a development aid to ensure the tests I write cover the
code I want/need them to. Just a few weeks ago we had an FHR shutdown
hang because of a typo that was getting hit in a (fortunately rare)
branch I thought we had test coverage for.

I want code coverage so I know when I'm working with unfamiliar code I
have an idea of how robust the test coverage is so I can adjust my patch
and review rigor appropriately.

I want code coverage so automation could potentially do things like warn
when changed lines in a patch aren't being tested - something that will
make me think twice about granting review.

I want code coverage to quantify how much our zeal to disable
intermittent failures is leading to reduced test coverage - something
that will snowball into increased technical debt and lead to lower
productivity.

I want code coverage because we can use it in automation to identify
what changes impact what tests, which potentially leads to us executing
a more minimal set of tests for changes.

Code coverage is a flashlight that illuminates a very dark alley of
JavaScript development today. I can't wait to have it so I can apply
more rigor to developing JavaScript. I also believe that good code
coverage tools can increase developer productivity and allow us to move
faster.

On behalf of JavaScript developers everywhere, I implore you to build
this functionality.

[Philipp Kewisch]

Post by Jason Orendorff
We're considering building a JavaScript API for dynamic analysis of JS
code.

I usually don't do this, but since the others have mentioned all the
good reasons and I am likewise totally excited about code coverage:

+1

[Joshua Cranmer 🐧]

Post by Jason Orendorff
We're considering building a JavaScript API for dynamic analysis of JS
code.
- Gather code coverage information (useful for testing/release mgmt?)

I've begged this several times, and, as I mentioned in another recent
thread, I've grown skeptical of any code coverage approach not based on
the JS runtime engine itself.

If you add only one new feature, this is the one you should add.

[Manu Sridharan]

Hello all,

I'm one of the maintainers of the Jalangi dynamic analysis framework for JavaScript:

https://github.com/SRA-SiliconValley/jalangi

Jalangi works via source-to-source transformation, and we already have an implementation of many of the clients you listed (e.g., record/replay, taint analysis, NaN detection). One of our key pain points when analyzing web apps is trying to instrument all loaded code, so having a supported API to do so would be hugely helpful. We would find a source-to-source API most useful, and I agree with Rob that supporting S2S is a good way to go in terms of maximizing flexibility for tool builders. Apart from source-to-source transformation, it would be useful to us to have a supported way to load some scripts at initialization time (in our case, the Jalangi runtime libraries), so that instrumented code can call into those scripts.

I will forward this thread to others who have worked on Jalangi to see if they have further feedback. We are highly supportive of this effort; I think a supported instrumentation API would make Firefox the browser of choice for those doing research on JavaScript dynamic analysis.

Best,
Manu

-------------------------
Manu Sridharan
Samsung Research America
http://manu.sridharan.net

Post by Jason Orendorff
We're considering building a JavaScript API for dynamic analysis of JS code.
- Gather code coverage information (useful for testing/release mgmt?)
- Trace all object mutation and method calls (useful for devtools?)
- Record/replay of JS execution (useful for devtools?)
- Implement taint analysis (useful for the security team or devtools?)
- Detect when a mathematical operation returns NaN (useful for game
developers?)
Note that the API would not directly offer all these features. Instead, it
would offer some powerful but mind-boggling way of instrumenting all JS
code. It would be up to you, the user, to configure the instrumentation, get
useful data out of it, and display or analyze it. There would be some
overhead
when you turn this on; we don't know how much yet.
We would present a detailed example of how to use the proposed API, but
we are
so early in the process that we're not even sure what it would look like.
There are several possibilities.
We need to know how to prioritize this work. We need to know what kind
of API
we should build. So we're looking for early adopters. If that's you, please
speak up and tell us how you'd like to instrument JS code.
--
Nicolas B. Pierron
Jason Orendorff
(JavaScript engine developers)

[Nicolas B. Pierron]

Hi,

Post by Manu Sridharan
https://github.com/SRA-SiliconValley/jalangi

I would be happy to discuss the way we can integrate such API in Firefox.

As currently we are not yet settled on one solution, We started a discussion
about Dynamic Analysis API on the mozilla.dev.tech.js-engine.internals
mailing list.

In this other thread, I evaluated Jalangi and why I think it does not fit
our needs at the moment. Feel free to reply to this other thread if you
want to discuss the technical aspect behind supporting such API. I would be
happy if we could share a common API for running analysis in an efficient
and non-intrusive manner.

[David Rajchenbach-Teller]

I would be interested in adding boundary checks and invariant checks
that could be eliminated in opt builds. Is this in the scope of your
project?

Post by Jason Orendorff
We're considering building a JavaScript API for dynamic analysis of JS code.
- Gather code coverage information (useful for testing/release mgmt?)
- Trace all object mutation and method calls (useful for devtools?)
- Record/replay of JS execution (useful for devtools?)
- Implement taint analysis (useful for the security team or devtools?)
- Detect when a mathematical operation returns NaN (useful for game
developers?)
Note that the API would not directly offer all these features. Instead, it
would offer some powerful but mind-boggling way of instrumenting all JS
code. It would be up to you, the user, to configure the instrumentation, get
useful data out of it, and display or analyze it. There would be some
overhead
when you turn this on; we don't know how much yet.
We would present a detailed example of how to use the proposed API, but
we are
so early in the process that we're not even sure what it would look like.
There are several possibilities.
We need to know how to prioritize this work. We need to know what kind
of API
we should build. So we're looking for early adopters. If that's you, please
speak up and tell us how you'd like to instrument JS code.

[Jason Orendorff]

Post by David Rajchenbach-Teller
I would be interested in adding boundary checks and invariant checks
that could be eliminated in opt builds. Is this in the scope of your
project?

No, I don't think so. S2S could certainly do it, but you wouldn't want
to use S2S on all browser code in opt builds. Too expensive.

But: there's already a pattern you can use to enable assertions that
will be zero-cost if your code runs in IonMonkey! I didn't realize this
until you mentioned it. But suppose you start out with this:

function DEBUG() {
return false;
}

function assert(condition) {
if (!condition) {
throw new Error("assertion failed");
}
}

Then you can write:

if (DEBUG()) assert(flowers.areBeautiful());

Now of course `assert(flowers.areBeautiful())` will not execute whether
IonMonkey kicks in or not, because DEBUG() returns false.

But when IonMonkey compiles this, it can do even better. First, DEBUG()
will be inlined. Then, since the return value is always `false`, the
whole if-statement will be optimized away. (Currently, we don't optimize
away if-statements that depend on global vars or even consts. Only
functions.)

Admittedly this pattern is not quite as pretty as
`assert(flowers.areBeautiful())`. Here's why that wouldn't work. JS
doesn't have macros. :( Arguments to a function are evaluated before the
function is called. This means that if we wrote assertions this way,
`flowers.areBeautiful()` would have to be called even in non-debug
builds. The language spec requires it. We need the if-statement to get
the right debug-only behavior.

-j

[Fitzgerald, Nick]

Post by Jason Orendorff

Post by David Rajchenbach-Teller
I would be interested in adding boundary checks and invariant checks
that could be eliminated in opt builds. Is this in the scope of your
project?

No, I don't think so. S2S could certainly do it, but you wouldn't want
to use S2S on all browser code in opt builds. Too expensive.
But: there's already a pattern you can use to enable assertions that
will be zero-cost if your code runs in IonMonkey! I didn't realize
function DEBUG() {
return false;
}
function assert(condition) {
if (!condition) {
throw new Error("assertion failed");
}
}
if (DEBUG()) assert(flowers.areBeautiful());
Now of course `assert(flowers.areBeautiful())` will not execute
whether IonMonkey kicks in or not, because DEBUG() returns false.
But when IonMonkey compiles this, it can do even better. First,
DEBUG() will be inlined. Then, since the return value is always
`false`, the whole if-statement will be optimized away. (Currently, we
don't optimize away if-statements that depend on global vars or even
consts. Only functions.)
Admittedly this pattern is not quite as pretty as
`assert(flowers.areBeautiful())`. Here's why that wouldn't work. JS
doesn't have macros. :( Arguments to a function are evaluated before
the function is called. This means that if we wrote assertions this
way, `flowers.areBeautiful()` would have to be called even in
non-debug builds. The language spec requires it. We need the
if-statement to get the right debug-only behavior.
-j
_______________________________________________
dev-platform mailing list
https://lists.mozilla.org/listinfo/dev-platform

I don't know if it is optimized by SpiderMonkey as well, but in the
debugger we have

|exports.dbg_assert =function dbg_assert(cond, e) {|
| if (!cond) {|
| return e;|
| }|
|}|


http://dxr.mozilla.org/mozilla-central/source/toolkit/devtools/DevToolsUtils.js#338

Which is then redefined inside of tests:

|// Redeclare dbg_assert with a fatal behavior.|
|function dbg_assert(cond, e) {|
| if (!cond) {|
| throw e;|
| }|
|}|


http://dxr.mozilla.org/mozilla-central/source/browser/devtools/debugger/test/head.js#60

The usage is a little nicer than requiring the conditional every time.

[Boris Zbarsky]

Post by Fitzgerald, Nick
I don't know if it is optimized by SpiderMonkey as well, but in the
debugger we have
|exports.dbg_assert =function dbg_assert(cond, e) {|

This requires evaluation of "cond" and "e" on the caller side,
unfortunately. That can't be optimized away.

-Boris

[Nicolas B. Pierron]

Post by Boris Zbarsky

Post by Fitzgerald, Nick
I don't know if it is optimized by SpiderMonkey as well, but in the
debugger we have
|exports.dbg_assert =function dbg_assert(cond, e) {|

This requires evaluation of "cond" and "e" on the caller side,
unfortunately. That can't be optimized away.

For function calls, yes. For non-effectful expressions, this is what
Recover Instructions (Bug 1003801) are made for.

By the way, "optimized away" is the title I chose for the blog post I am
writing to explain recover instructions :D

[Nicolas B. Pierron]

Post by David Rajchenbach-Teller
I would be interested in adding boundary checks and invariant checks
that could be eliminated in opt builds. Is this in the scope of your
project?

From an analysis point of view I am not sure to see how useful such
information would be. It seems that you are more interested in having
assertions where you can obtain a backtrace than having an analysis which
does not change the behavior of the program.

[m***@public.gmane.org]

This is music to my ears :)

Especially the combination of taint analysis with code coverage would be extremely helpful. The main drawback of dynamic security testing is the uncertainty how reliable/complete the achieved results are (opposed to static analysis, where you know exactly how much code was analysed).

[Sylvestre Ledru]

Post by Jason Orendorff
We're considering building a JavaScript API for dynamic analysis of JS code.
- Gather code coverage information (useful for testing/release mgmt?)

Yes, I confirm that we would be happy to get a clean and efficient way
to instrument the Firefox and Firefox OS Javascript code.

Ultimately, in order to aggregate the result with the C/C++ coverage,
we would also need this API to be able to export the coverage results
into a "standard" format (gcna/gcdo).

Thanks for this promising proposal,
Sylvestre

[Joshua Cranmer 🐧]

Post by Sylvestre Ledru

Post by Jason Orendorff
We're considering building a JavaScript API for dynamic analysis of JS code.
- Gather code coverage information (useful for testing/release mgmt?)

Yes, I confirm that we would be happy to get a clean and efficient way
to instrument the Firefox and Firefox OS Javascript code.
Ultimately, in order to aggregate the result with the C/C++ coverage,
we would also need this API to be able to export the coverage results
into a "standard" format (gcna/gcdo).

Ew, no. Don't use gcna/gcdo format--they're incredibly wonky and
difficult to read independently. Use something like LCOV's files instead.

[Fitzgerald, Nick]

Post by Jason Orendorff
We're considering building a JavaScript API for dynamic analysis of JS code.
- Gather code coverage information (useful for testing/release mgmt?)
- Trace all object mutation and method calls (useful for devtools?)
- Record/replay of JS execution (useful for devtools?)
- Implement taint analysis (useful for the security team or devtools?)
- Detect when a mathematical operation returns NaN (useful for game
developers?)
Note that the API would not directly offer all these features.
Instead, it
would offer some powerful but mind-boggling way of instrumenting all JS
code. It would be up to you, the user, to configure the
instrumentation, get
useful data out of it, and display or analyze it. There would be some
overhead
when you turn this on; we don't know how much yet.
We would present a detailed example of how to use the proposed API,
but we are
so early in the process that we're not even sure what it would look like.
There are several possibilities.
We need to know how to prioritize this work. We need to know what kind
of API
we should build. So we're looking for early adopters. If that's you, please
speak up and tell us how you'd like to instrument JS code.

FWIW, I just remembered that Honza and the Firebug folks have been
asking for something like this for a while, and they can't completely
remove dependency on the old debugger API until there is a replacement.

https://bugzilla.mozilla.org/show_bug.cgi?id=797876

[Erik Rose]

Post by Jason Orendorff
We're considering building a JavaScript API for dynamic analysis of JS code.

I'd love to add dynamic JS analysis to DXR's quiver. Static JS analysis is coming in early fall, and my intern Marcell is hard at work wringing what he can out of pointer analysis. Of course, that sort of thing is necessarily fuzzy in such a dynamic language, so we'd welcome the information a dynamic pass would add.

Being able to, say, record the property sets of objects thrown around during automated tests would boost our confidence about the type signatures of functions (such as they are) and make our query results more useful. Being able to find where undefineds or NaNs pop out of running code would make it easier to spotlight ones which are spurious. We'd love to have the hooks!

--
Erik Rose
DXR Lead

[Neil]

Honza and the Firebug folks have been asking for something like this
for a while, and they can't completely remove dependency on the old
debugger API until there is a replacement.
https://bugzilla.mozilla.org/show_bug.cgi?id=797876

JSD1 just got removed in bug 800200, or does not include the profiler?

[Philip Chee]

Post by Fitzgerald, Nick
FWIW, I just remembered that Honza and the Firebug folks have been
asking for something like this for a while, and they can't completely
remove dependency on the old debugger API until there is a replacement.
https://bugzilla.mozilla.org/show_bug.cgi?id=797876

Given that the old debugger API was just removed from the tree, it looks
like Firebug is S.O.L.

Phil

[Jonathan Griffin]

The A-team would be very interested in being able to track JS code
coverage; if you implemented the ability, we could add jobs in TBPL to
track our test coverage over time, which would probably be useful and
interesting.

We'd be happy to get involved at any stage where it's practical to start
thinking about how we could integrate this into test runs.

Jonathan

Post by Jason Orendorff
We're considering building a JavaScript API for dynamic analysis of JS code.
- Gather code coverage information (useful for testing/release mgmt?)
- Trace all object mutation and method calls (useful for devtools?)
- Record/replay of JS execution (useful for devtools?)
- Implement taint analysis (useful for the security team or devtools?)
- Detect when a mathematical operation returns NaN (useful for game
developers?)
Note that the API would not directly offer all these features.
Instead, it
would offer some powerful but mind-boggling way of instrumenting all JS
code. It would be up to you, the user, to configure the
instrumentation, get
useful data out of it, and display or analyze it. There would be some
overhead
when you turn this on; we don't know how much yet.
We would present a detailed example of how to use the proposed API,
but we are
so early in the process that we're not even sure what it would look like.
There are several possibilities.
We need to know how to prioritize this work. We need to know what kind
of API
we should build. So we're looking for early adopters. If that's you, please
speak up and tell us how you'd like to instrument JS code.