my pull requests and further branches/work

[User4martin]

This is to serve as an overview for the changes/pullrequests that I made.

---

refactor-generate-source

basic refactor: move functionality into scanLine. Making it easier for future plugins to change behaviour by overwriting scanLine.
This is continued in refactor-generate-source-full-tokens.

It also contains some changes that speed up compilation. Some of them could be implemented independent of the refactor.

refactor-generate-source-full-tokens

Scanline is now replaced by

• scanTextLine: function (line)
• scanTagLine: function (openTag, line, closeTag)

literals are handled before passing to those function.

As a result text is no longer split at literals, and both functions are always called with the full tag between <%...%> ( or %>...<% for scanTextLine)

For templates containing literal <%% this also results in less __append in the compiled output, and therefore a speed up of the generated function.

Having less calls to scanTagLine has a positive effect on compilation speed.

Due to the current handling of unmatched close tags, and literals, the regex got more complicated.

The new scanTagLine requiring openTag and closeTag,also means that it will be harder to implement stand-alone tags or nested tags.
E.g. a literal section <%[ now all is literal %> and <% are just text ]%>
will not work. Because the tags are expected in pairs.

If that is a concern then lets look at alternatives. From my various attempts on changing the main loop, I expect that passing the tokens individually will come with a slow down of compilation).

A plugin could overwrite the entire generateSource function, which is the loop that splits the text into tokens.

On the other hand, being guaranteed the full content of each tag makes other tasks easier.

• <%% can now appear within code. That was broken before
• that also enables (old style) <% include 'foo<%%bar' %> (filename contains <% )

This also depends on #284

refacter-preview-next-token

changes scanTextLine: function (line)
to scanTextLine: function (line, prevCloseTag, nextOpenTag)

Passing the tags in, allows the function to handle the space slurping.

This also fixes the hardcoded regex for <%_ which did non use opts.delimiter.

optimize_combine_append_calls

Speed up (in none compileDebug) for the generated functions.

In the generated code, consecutive __append calls are merged to __append(a,b,c)

Less calls => faster.

This helps for templates with many <%- / <%= tags. Since those and the html in between them all use __append.

Templates with a lot of <% if (...) %> tags, will not benefit as much.

[User4martin]

timings current master vs optimize_combine_append_calls

Master:

git checkout master
Switched to branch 'master'
node bench-ejs.js -r 7    --compile --cache
Running avg accross:  7
name:                          avg        med        med/avg    min        max                  loops
single tmpl compile:           1.525      1.513      1.513      1.481      1.633                20000
single tmpl compile (debug):   1.744      1.725      1.734      1.718      1.811                20000
large tmpl compile:            5.947      5.822      5.813      5.571      6.619                 1000   <<<<
include-1 compile:             1.054      1.054      1.053      1.033      1.084                20000
-
single tmpl cached:            1.188      1.201      1.191      1.119      1.28                 50000
single tmpl cached (debug):    1.186      1.194      1.192      1.158      1.201                50000
large tmpl cached:             4.077      4.038      4.046      3.971      4.289                 4000   <<<<
include-1 cached:              1.922      1.917      1.915      1.868      2.032                20000
include-2 cached:              2.325      2.231      2.238      2.073      2.729                20000
locals tmpl cached "with":     3.265      3.274      3.27       3.176      3.37                 30000
locals tmpl cached NO-"with":  2.102      2.103      2.099      2.085      2.134                30000
-

optimize-combine-append-calls

git checkout optimize-combine-append-calls
Switched to branch 'optimize-combine-append-calls'
Your branch is up-to-date with 'origin/optimize-combine-append-calls'.
node bench-ejs.js -r 7    --compile --cache
Running avg accross:  7
name:                          avg        med        med/avg    min        max                  loops
single tmpl compile:           1.057      1.038      1.04       1.019      1.151                20000
single tmpl compile (debug):   1.248      1.228      1.234      1.221      1.338                20000
large tmpl compile:            2.776      2.724      2.729      2.674      2.959                 1000   <<<<
include-1 compile:             0.853      0.853      0.853      0.85       0.858                20000
-
single tmpl cached:            0.858      0.853      0.85       0.816      0.902                50000
single tmpl cached (debug):    0.932      0.935      0.931      0.907      0.959                50000
large tmpl cached:             1.528      1.493      1.494      1.478      1.649                 4000   <<<<
include-1 cached:              1.823      1.78       1.78       1.756      2.028                20000
include-2 cached:              1.933      1.922      1.916      1.885      2.034                20000
locals tmpl cached "with":     1.347      1.345      1.344      1.321      1.38                 30000
locals tmpl cached NO-"with":  0.901      0.892      0.892      0.88       0.954                30000

the runtime (cached) performance of "large tmpl" is in huge parts because the template has many <%%, and they no longer split the text into many __append.

Compared to

git checkout  refactor_preview_next_token
Switched to branch 'refactor_preview_next_token'
Your branch is up-to-date with 'origin/refactor_preview_next_token'.
node b/bench-ejs.js -r 7     --cache
single tmpl cached:            1.051      0.981      0.979      0.958      1.269                50000
single tmpl cached (debug):    1.004      0.993      1.006      0.939      1.068                50000
large tmpl cached:             2.522      2.489      2.499      2.371      2.778                 4000   <<<<
include-1 cached:              1.966      1.985      1.965      1.826      2.132                20000
include-2 cached:              2.035      2.029      2.032      1.952      2.113                20000
locals tmpl cached "with":     2.018      2.01       2.015      1.998      2.04                 30000
locals tmpl cached NO-"with":  1.322      1.324      1.321      1.309      1.348                30000

refactor_preview_next_token also has NO split for <%%. But it does NOT join other __append calls

[User4martin]

plugin-snippets

see #274

optimizations

Added a cache for the fs.fileExists call (obeying opts.cache)

Also added a key to module exports to allow hooking fileExists (same as for loading a file)

Made a small optimization to regex, small speed gain.

plugin-ejs-locals

see https://github.com/RandomEtc/ejs-locals

This is used by some frameworks. And if not available can prevent upgrading.

I had to expose some more functions in ejs.

exports.getIncludePath = getIncludePath;
exports.handleCache = handleCache;
exports.includeFile = includeFile;

If this is not desired, I can copy them to the plugin.

plugin-opts-filter

For people still stuck on express 3 or anything using the old (IIRC 3 arg) renderFile.

It can happen that locals and options get mixed up.
In this case passing in a locals "client" (not uncommon), will be seen as option.

This plugin allows to filter them out.