=head1 NAME

PIL2JS - PIL to JavaScript compiler

=head1 DESCRIPTION

C<pil2js.pl> is a Perl 5 program which takes PIL as input (as given by C<pugs
-CPIL1-Perl5>) and outputs JavaScript.

=head1 USAGE

There's an interactive shell, F<jspugs.pl>. Simply start F<jspugs.pl>, then
precompile the Prelude by entering C<:precomp>. Now enter some Perl 6 code and
point your browser to F<./output.html> or prefix your expression with C<:e> if
you want it to be evaluated outside the browser using Spidermonkey's F<bin/js>.

Additionally, there's F<./runjs.pl>, which accepts the same options as Pugs.
It will first compile your code to JavaScript and then run it using
Spidermonkey's F<bin/js>:

  $ ./runjs.pl -e 'say "Hello, World!"'

Alternatively, you can use the compiler directly:

  $ cd perl5/PIL2JS
  $ ./pil2js.pl -o Prelude.js lib6/Prelude/JS.pm  # Precompile the Prelude
  $ ./pil2js.pl -o output.js input.pl             # Compile the actual prog
  $ ./pil2js.pl \                                 # Generate a .html
      --link=html \
      -o output.html \
      ~METAMODEL ~libjs/PIL2JS.js ~Prelude.js input.js
  # This links the Perl 6 MetaModel, the basic JS Prelude (PIL2JS.js), the
  # Perl 6 Prelude (Prelude.js), and your code together, producing output.html.

=head1 SMOKING

You can use C<make smoke-js> to compile the whole testsuite to JavaScript and
run it using Spidermonkey's F<bin/js>. The test results will be outputted as
F<smoke-js.html>.

=head1 FILE HIERARCHY

  .
  |-- pil2js.pl           Frontend for the PIL:: modules
  |-- runjs.pl            Compiles Perl 6 to JavaScript and runs it
  |-- jspugs.pl           Interactive shell for PIL2JS development
  |-- lib
  |   |-- PIL2JS.pm       Perl 5 wrapper module for pil2js.pl, runjs.pl, etc.
  |   |-- PIL.pm          Main compiler modules, uses all other PIL:: modules
  |   |-- PIL
  |   |   |-- Parser.pm   PIL::Parser (parses the output of -CPIL1-Perl5)
  |   |   `-- <PIL nodetype>.pm
  |   |                   Backend compiler modules
  |   |-- Prelude
  |   |-- `-- JS.pm       Part of the JavaScript Prelude,
  |   |                   written in Perl 5 using P5Macros
  |   |-- Perl6
  |   |   `-- Run
  |   |       `-- JS.pm   Perl 5 module allowing easy integration of
  |   |                   Perl 6 code which will be run by JSSM
  |-- lib6
  |   |-- Prelude
  |   |   |-- JS.pm       Part of the JavaScript Prelude,
  |   |   |               written in Perl 6 with mixed JS
  |   |   `-- JS/         Submodules of the Prelude (e.g. Prelude::JS::Array,
  |   |                   Prelude::JS::ControlFlow, etc.)
  |-- libjs
  |   |-- PIL2JS.js       Part of the JavaScript Prelude,
  |   |                   written in JavaScript
  |   `-- Perl6/          JavaScript Perl 6 Metamodel

=head1 WHAT'S WORKING ALREADY?

=over

=item *

Variables (scalars, arrays, and hashes), assignment, binding

=item *

Subroutine and (slightly simplified) method declaration (both global and
lexical declarations)

=item *

Subroutine and method invocation, including optional and named parameters,
slurpy scalars, arrays, and hashes, C<< &prefix:<*> >> (e.g.
C<foo(*@parameters)>), C<is rw>, C<is readonly>, and C<is copy>

=item *

Basic operators (C<< &infix:<==> >>, C<< &infix:<eq> >>, C<< &prefix:<+> >>,
C<< &infix:<=:=> >>, C<< &infix:<===> >>...)

=item *

Basic builtins (C<say>, C<defined>, C<if>, C<loop>, C<while>, C<until>, ...)

=item *

Primitives which change the control flow (C<return>, C<last>, C<next>, C<redo>,
C<die>, C<try>, ...)

=item *

Container semantics, binding, autodereferencing references

=item *

C<Proxy> class (but not yet user-visible, as support for class methods is still
lacking)

=item *

Array and hash autovivification

=item *

Stub OO builtins (C<ref>, C<isa>), awaiting C<-CPIL2> to fully support Stevan's
excellent MetaModel (see F<perl5/Perl6.MetaModel/>)

=item *

Perl 5-style regular expressions

=item *

Global C<JS> namespace to use native JavaScript classes and functions (e.g.
C<$*JS::document.write>, C<JS::alert "Pugs rocks">)

=item *

JSAN integration

=back

=head1 DESIGN

=head2 Containers

Because JavaScript passes primitive values (strings, numbers, etc.) by value, I
had to create a proxying object, as Perl requires support for C<is rw> and C<is
copy>. C<new PIL2JS.Box(value)> returns an object supporting the methods
C<.FETCH()> (returns C<value>), C<.STORE(new_value)> (set to new value),
C<.clone()>, and C<.toNative()> (return value as native object).

Additionally, there's C<PIL2JS.Box.ReadOnly>, which proxies a C<PIL2JS.Box>,
but C<die()>s upon C<.STORE()>. This is among other things needed for readonly
subroutine parameters (e.g. C<sub foo ($x) {...}> instead of C<sub foo ($x is
rw) {...}>). C<PIL2JS.Box.ReadOnly>s can still be rebound, though, but
rebinding does not affect the original variable.

C<new PIL2JS.Box.Constant(value)> creates a constant box, meaning both
C<.STORE()> and C<.BINDTO()> will fail. Amongst others, literals (C<42>, C<"a
string">, C<sub {...}>) are serialized to C<PIL2JS.Box.Constant>s.

=head3 Container Types

All C<PIL2JS.Box>es have an associated container type
(C<PIL2JS.ContainerType.Scalar>,
C<PIL2JS.ContainerType.Array>, or C<PIL2JS.ContainerType.Hash>) which can't be
changed later on. This also means that a C<Scalar> container may never hold an
C<Array>, but only I<references> to arrays.

"What about the C<&> sigil?" -- C<&code_variables> are stored in C<Scalar>
containers exactly like ordinary scalar values. The only requirement (which is,
BTW, not enforced currently) is that you are not allowed to store things in
C<&>-sigilled variables which do not inherit from C<Code> (i.e., C<&x = 3> is
bogus).

=head3 Assignment and binding

=head4 Assignment

Assignment is, thanks to the container type C<PIL2JS.Box>, simple:
C<dest.STORE(source)> (C<.STORE> will call C<source.FETCH()> and return
C<source>, allowing constructs like C<($a = $b) = $c>, which has the same
effect as C<$a = $b; $a = $c>.)

=head4 Binding

Binding is similar: C<dest.BINDTO(source)>

We can't just use C<dest = source>, because C<dest> might be the result of some
subroutine, e.g. C<@array[$idx] := $foo>, while really is
C<< @array.postcircumfix:<{ }>($idx) := $foo >>.

=head3 Identity comparision

The C<.uid> property which every C<PIL2JS.Box> object has got contains the
unique ID of the box (or C<undefined> if it's a C<PIL2JS.Box.Constant>, as we
can't map all real numbers and more to a finite C<.uid>). C<< &infix:<=:=> >>
can then compare C<this.uid> with C<other.uid>.

=head3 Aggregates

Aggregates require some extra work beyond what's already needed for scalars:
Arrays and hashes (but not lists!) require reboxing of their
soon-to-be-elements to be readwrite, e.g. the following should work:

  my @a = (1,2,3);
  @a[1]++;  # should work even though 2 is a constant

Previously, this reboxing was incorrectly done by C<< &infix:<,> >> (the list
constructor sub (C<(1,2,3)> really means C<< &infix:<,>(1,2,3) >>)), but this
functionality really has to stay in the implementation of C<.STORE> of array
and hash containers.

=head4 List magic

Lists have a number of additional properties (which, BTW, explains the length
of C<< &infix:<,> >>'s implementation (in C<Prelude::JS::Array>)):

  ($a, $b) = ($foo, $bar);  # same as
  $a = $foo; $b = $bar;

  ($first, @rest) = foo();  # same as
  {
    my @temp_array  = foo();
    $first         := shift @temp_array;
    @rest          := @temp_array;
  }

  ($a, $b) = ($b, $a);      # same as
  {
    my $backup_of_a = $a;
    $a = $b;
    $b = $backup_of_a;
  }

  ($a, $b) := ($b, $a);     # same as
  {
    my $backup_of_a := $a;
    $a := $b;
    $b := $backup_of_a;
  }

=head2 Calling conventions

Because Perl's subroutine signatures are much richer than JavaScript's ones, I
had to invent own calling conventions.

All functions take exactly one JavaScript-argument, a native JavaScript
C<Array> (much like Perl 5's C<@_>). Then the individual arguments are
extracted (see C<PIL::Nodes>, package C<PIL::Params>).

Additionally to the arguments passed by the user, the first argument is always
an unboxed C<PIL2JS.Context> object. Methods expect C<$?SELF> as their second
argument.

The last argument is always the return continuation.

=head3 Flattening (C<< &prefix:<*> >>)

  # Perl 6
  foo [,] @a, $x, @b;

  # PIL
  foo(&prefix:<*>(@a), $x, &prefix:<*>(@b));

C<< &prefix:<*> >> is a special subroutine which returns its input argument
(always an array) with the special C<flatten_me> JavaScript property set (you
could think of it as C<< @input_array but PIL2JS::Internals::flatten_me >>).

Subroutines then search their input arguments for arrays which have the
C<flatten_me> property set and handle accordingly (see
C<PIL2JS.possibly_flatten> in F<libjs/PIL2JS.js>).

=head2 Full continuations

Because JavaScript doesn't natively support full continuations, I had to fake
them:

  foo(); bar(3, baz());   # is compiled as
  foo(-> () {
    baz(-> $arg {
      bar(3, $arg);
    });
  });

If a sub wants to return, it simply executes the return continuation given as
the last argument.

Unfortunately, simply calling the return continuation causes stack
overflow errors, as the original functions are, from a JavaScript point of
view, never left. Therefore, instead of calling the return continuation
directly, an exception is thrown:

  // Instead of
  continuation(retval);
  // an exception is thrown:
  throw function () { continuation(retval) };

This makes sure all functions are, from a JavaScript standpoint seen, really
left. C<PIL2JS.runloop> catches those exceptions and runs them:

  PIL2JS.runloop(function () {
    ...code...
  });

To convert a function from CPS to normal style, you can use
C<PIL2JS.cps2normal>:

  var retval = PIL2JS.cps2normal(unboxed_function, [...args...]);

Note that this breaks, of course, call/cc magic. This means that coroutines,
C<&?CALLER_CONTINUATION>, etc. might not work correctly. The workaround is
easy, luckily:

  # Instead of (&statement_control:<if> uses PIL2JS.cps2normal)...
  if some_coroutine() {...}
  
  # ...use:
  my $retval = some_coroutine();
  if $retval {...}

C<&next>, C<&last>, and C<&redo> will be converted to use continuations,
but currently they still use exceptions to escape.

=head3 Coroutines

With full continuations, coroutines are relatively easy to implement. All
coroutines maintain a pointer to their entrypoint (a continuation) in the
global array C<PIL2JS.coro_entrypoints>, index by a (allocated at compile-time)
ID. Upon invocation, they simply call their entrypoint.

A call to C<&yield> updates the entrypoint accordingly, C<&return> resets it.

=head3 Speed considerations

Because we have to pass many closures around and throw many exceptions (one for
each C<return()>), speed suffers, of course. But sadly, there isn't much PIL2JS
can do against this, it should be noted though that, if JavaScript provided
C<goto>, it is expected that PIL2JS would be at least twice as fast as it is
now. PIL2JS would then, combined with Opera's fast JavaScript engine,
outperform Pugs's old Haskell runcore easily.

=head2 Metamodel

I use Stevan's C<Perl6.MetaModel> as the library providing classes like
C<Perl6.Class>, C<Perl6.MetaClass>, etc.

Note that I do not store the actual method bodies in the classes; Instead, all
methods merely return the boxed sub object (a C<PIL2JS.Box.Constant>) upon
invocation. I.e. the way to call a method is
C<classobj.FETCH().can(methname).call("__i_am_pil2js").FETCH().([...])>. The
C<__i_am_pil2js> guard is there so other code (which doesn't know about boxed
subs) doesn't accidentally call our methods.

Note that our methods do I<not> get a C<Perl6.Instance> as C<$?SELF> (this is
due to perfomance reasons) -- however, when calling a method, the metamodel
does get used, so this decision is hidden from the end user.

=head2 JavaScript integration

The support for calling native JavaScript functions is quite extensive.

=head3 Global C<*JS> Perl namespace

The global C<*JS> namespace can be used to call native JavaScript functions.
E.g.:

  JS::alert "Hi!";
  # calls alert with a native JS string

  $JS::document.write("Hi!");
  # calls document.write with a native JS string

You can even use JavaScript's higher order functions or store native functions
in variables:

  $JS::window.setTimeout(&subref, $delay);
  my $alarm = &JS::alert;

=head3 C<&JS::inline> compile-time macro

Additionally, there's the C<&JS::inline> compile-time macro. It expects a
string as its only argument to inline into the resulting JavaScript code.

  say 4 + JS::inline('new PIL2JS.Box(3)');
  # 7

You can't pass arbitrary expressions to C<&JS::inline>, though, i.e. the
following is invalid:

  say 4 + JS::inline("new PIL2JS.Box($num)");  # does not work

You have to use a C<BEGIN> block to circumvent this limitation:

  say 4 + JS::inline(BEGIN { "new PIL2JS.Box($num)" });  # works

If you read through the various C<Prelude::JS::*> modules, you'll recognize
the following idiom:

  # We pass boxed variables to an unboxed native JavaScript function,
  # PIL2JS takes care of the necessary converting.
  sub p6sub ($p6arg1, $p6arg2) {
    JS::inline('(function (jsarg1, jsarg2) {
      return jsfunc(jsarg1, jsarg2);
    })')($p6arg1, $p6arg2);
  }

  # Alternatively, we use a boxed sub. This means we have to manually convert,
  # but the container bindings etc. don't get lost. Also note that we have to
  # call the current continuation to return, *not* JavaScript's "return".
  sub p6sub ($p6arg1, $p6arg2) {
    # We pass boxed variables to an unboxed native JavaScript function,
    # PIL2JS takes care of the necessary converting.
    JS::inline('new PIL2JS.Box.Constant(function (jsarg1, jsarg2) {
      var cxt    = args.shift(), cc     = args.pop();
      var jsarg1 = args[0],      jsarg2 = args[1];
      cc(new PIL2JS.Box.Constant(jsfunc(jsarg1.FETCH(), jsarg2.FETCH())));
    })')($p6arg1, $p6arg2);
  }

=head2 JSAN integration

C<Pugs.Parser> parses C<use jsan:Foo> statements and emits a call to an
internal PIL2JS sub:

  use jsan:Foo <bar baz>;   # is really
  PIL2JS::Internals::use_jsan_module_imp("Foo", <bar baz>);

C<&PIL2JS::Internals::use_jsan_module_imp> then asks the C<JSAN> object to load
C<Foo>. Furthermore, it autoboxes and autorenames all functions the module
exports. This means, that the following works:

  use jsan:Number.Roman <to_roman>;
  say to_roman 42;

  # Without the autoboxing and autorenaming, you'd have to write:
  say JS::to_roman 42;

Note that the JSAN integration is not available when running under
Spidermonkey's F<bin/js>; This is not a bug in PIL2JS or JSAN, it's because
F<bin/js> can't download arbitrary URLs.

=head3 Usage

  # Compile the Perl 6 to PIL
  $ pugs -CPIL1-Perl5 -we '
      $*JS::JSAN.addRepository("/tmp/jsanlibs");
      use jsan:Number.Roman <to_roman>;
      say "42: {to_roman 42}";
    ' > /tmp/test.pil
  
  # Compile the PIL to JavaScript
  $ ./pil2js.pl -o /tmp/test.js /tmp/test.pil
   
  # Link the JavaScript, JSAN, the Metamodel and the Prelude together
  $ ./pil2js --link=html -o /tmp/test.html \
      ~http://openjsan.org/src/c/cw/cwest/JSAN-0.10/lib/JSAN.js \
      ~METAMODEL \
      ~$PWD/libjs/PIL2JS.js ~$PWD/Prelude.js /tmp/test.js
  
  # Fetch Number.Roman from JSAN and copy its lib/* to /tmp/jsanlibs
  # Point your $BROWSER to test.html

=head2 Macros written in Perl 5

Similar as what is possible in PIL-Run, PIL2JS allows you to write macros in
Perl 5. These kind of macros are passed (already compiled to JS) parameters and
return modified JS code. This allows to inline sub definitions in some cases:

    # Input Perl 6 code
    if $cond { $then() }

    # Perl 6 code which would be emitted without P5Macros
    &statement_control:<if>($cond, $then, {});

    # Perl 6 code which is emitted with P5Macros
    if $cond { $then() } else { {}() }

=head2 Regular expressions

=head3 Perl 5-style regexes

Supporting Perl 5-style regexes (C<rx:Perl5/.../>) is only bookkeeping work, as
JavaScript provides a C<RegExp> class which is able to match against Perl 5-style
regexes.

=head3 Perl 6's rules

Supporting Perl 6's rules is much harder, as JavaScript doesn't provide a
native class which could be used to match against Perl 6 regexes. There're two
solutions to this problem:

=over

=item *

We write a native JavaScript library which'd be capable of matching against
Perl 6's rules.

=item *

We write a rule matching engine in Perl 6 (or any other language which we can
compile to JavaScript) and compile it to JavaScript.

=back

=head2 Laziness

=head3 C<lazy {...}>

PIL2JS fully supports nothingmuch's accepted C<lazy {...}> proposal (see
F<t/var/lazy.t>). Only one real problem arose during implementing C<&lazy>:
C<.FETCH> is called very often, for example at sub return (because
C<PIL2JS.Box.ReadOnly> needs to find out the container type of the thing it'll
proxy -- and this operation requires a C<.FETCH>). Therefore, C<&lazy>'s
current implementation does not return the lazy value directly, but an
autodereffing reference to it. This way the body is only called when it is
really needed.

=head3 Lazy lists

Lazy lists aren't supported at all yet, you may want to look at PIL-Run if you
want to experiment with lazy lists.

=head1 JAVASCRIPT RUNTIME ENVIRONMENTS

PIL2JS does not generate code depending on a special JavaScript runtime or
version, but PIL2JS's accompanying scripts and modules provide special support
for SpiderMonkey.

=head2 SpiderMonkey

PIL2JS supports Mozilla's
L<SpiderMonkey|http://www.mozilla.org/js/spidermonkey/> (written in C) in three
ways:

=over

=item *

The compiled code is given to SpiderMonkey's F<bin/js>. This approach does not
need any special Perl 5 modules, but it suffers from a buffering problem and it
seems to need much more RAM than the other engines.

=item *

There's the C<JavaScript::SpiderMonkey> Perl 5 module (also named C<JSSM>)
which embeds allows embedding of JavaScript code in Perl 5 applications. It
does not suffer from F<bin/js>'s buffering problems and it seems that,
curiously, it doesn't need as much RAM as F<bin/js>, but the installation of
C<JSSM> is a bit tricky.

=item *

There's the C<JavaScript> Perl 5 module (also named C<JSPM>) which also
exports perl5 functions that implements some primitives.  use

  runjs.pl --run=jspm --perl5

to enable it.

There is also a script C<util/jsperl> then can be used directly.

You probably need snapshot releases of C<JavaScript> modules for this to work.
L<http://home.clkao.org/~clkao/jspm>

=back

=head2 Opera's JavaScript engine

Opera's JavaScript engine outperforms SpiderMonkey, but sadly it doesn't
provide a console application for running JavaScript code. (If it did, PIL2JS
would probably be approximately twice as fast as it is when using SpiderMonkey.)

Additionally, Opera displays the stack backtrace along with exceptions, so it
might be more useful for debugging PIL2JS problems.

=head1 TODO

=over

=item *

Primitives!

=item *

Don't call all subs in list context -- but to fix this reliably, C<-CPIL1-Perl5> has
to give better context information.

=item *

C<method foo (Bar $self:)> should create a C<&Bar::foo>, not a C<&Main::foo>.
Required to implement hash and array autovification elegantly. (Partly a PIL1
bug.)

=item *

More OO and meta stuff (but requires PIL2 or fixes to PIL1).

=item *

Short term goals:

=over

=item *

Fix the P5Macros C<< &statement_control:<if> >> and C<<
&statement_control:<unless> >> to preserve lvaluehood.

=item *

Fix a problem with C<pad["$_"]> being stomped in combination with rules stuff
(C<int.t> doesn't pass 100% because of this).

=item *

Only make vars declared by C<env> accessible via C<$CALLER::> (according to
newest S02).

=item *

Make C<&?SUB> available in anonymous subs as well (C<my sub> counts as
anonymous).

=back

=back

=head1 AUTHOR

Ingo Blechschmidt C<< <iblech@web.de> >>

=head1 LICENSE

This program is free software; you can redistribute it and/or modify it under
the same terms as Perl itself. See L<perlgpl> and L<perlartistic> for details.

=cut