multiple duplicates of node resources + source codes

[SailorMax]

• VSCode Version: 1.14.0
• OS Version: Windows 10

Can you add some kind of control to reduce duplicates of node modules inside VSCode?
Plus do not put source codes and development examples to release, if it doesn't require.
#28874 was closed => I create new more global bug report.

Duplicates module name	Where is copy	How many files/size we can save
nan	node_modules/, nsfw, node-pty	90 files / 1MB
graceful-fs	node_modules/, nsfw	7 files / 0.031MB
applicationinsights	extensions: git, html, json, markdown, merge-conflict, typescript	230 files / 0.750MB
vscode-extension-telemetry	extensions: git, html, json, markdown, merge-conflict, typescript	30 files / 0.075MB
vscode-jsonrpc	extensions: css (2x), html (2x), json (2x)	110 files / 0.550MB
vscode-languageclient	extensions: css, html, json	60 files / 0.420MB
vscode-languageserver-types	extensions: css (2x), html (2x), json (2x)	25 files / 0.370MB
vscode-nls	extensions: css, configuration-editing, extension-editing, git, grunt, gulp, html, jake, javascript, json, markdown, merge-conflict, ms-vscode.node-debug, npm, php, typescript	90 files / 0.225MB
winreg	extensions: git, html, json, markdown, merge-conflict, typescript	30 files / 0.260MB
jsonc-parser	extensions: configuration-editing, extension-editing, javascript, json	18 files / 0.177MB
argparse	extensions: extension-editing, markdown	27 files / 0.126MB
entities	extensions: extension-editing, markdown	13 files / 0.058MB
linkify-it	extensions: extension-editing, markdown	6 files / 0.041MB
markdown-it	extensions: extension-editing, markdown	59 files / 0.556MB
mdurl	extensions: extension-editing, markdown	9 files / 0.027MB
sprintf-js	extensions: extension-editing, markdown	13 files / 0.036MB
uc.micro	extensions: extension-editing, markdown	9 files / 0.006MB
iconv-lite	node_modules/, extensions/git	26 files / 0.342MB
agent-base	node_modules/, extensions: javascript, json, ms-vscode.node-debug, ms-vscode.node-debug2	28 files / 0.076MB
debug	node_modules/, extensions: javascript, json, ms-vscode.node-debug, ms-vscode.node-debug2	44 files / 0.136MB
extend	node_modules/, javascript, json, ms-vscode.node-debug, ms-vscode.node-debug2	24 files / 0.052MB
http-proxy-agent	node_modules/, extensions: javascript, json, ms-vscode.node-debug, ms-vscode.node-debug2	28 files / 0.096MB
https-proxy-agent	node_modules/, extensions: javascript, json, ms-vscode.node-debug, ms-vscode.node-debug2	28 files / 0.124MB
ms	node_modules/, extensions: javascript, json, ms-vscode.node-debug, ms-vscode.node-debug2	20 files / 0.032MB
request-light	extensions: javascript, json, ms-vscode.node-debug, ms-vscode.node-debug2	18 files / 0.045MB
mdurl	extensions: extension-editing, markdown	9 files / 0.027MB
balanced-match	node_modules/, node_modules/nsfw, extensions: ms-vscode.node-debug, ms-vscode.node-debug2	12 files / 0.033MB
brace-expansion	node_modules/, node_modules/nsfw, extensions: ms-vscode.node-debug, ms-vscode.node-debug2	12 files / 0.042MB
concat-map	node_modules/, node_modules/nsfw, extensions: ms-vscode.node-debug, ms-vscode.node-debug2	15 files / 0.021MB
glob	extensions: ms-vscode.node-debug, ms-vscode.node-debug2	7 files / 0.072MB
inflight	node_modules/nsfw, extensions: ms-vscode.node-debug, ms-vscode.node-debug2	8 files / 0.012MB
inherits	node_modules/, node_modules/nsfw, extensions: ms-vscode.node-debug, ms-vscode.node-debug2	15 files / 0.024MB
minimatch	node_modules/, node_modules/nsfw, extensions: ms-vscode.node-debug, ms-vscode.node-debug2	12 files / 0.124MB
once	node_modules/nsfw, extensions: ms-vscode.node-debug, ms-vscode.node-debug2	8 files / 0.014MB
path-is-absolute	node_modules/nsfw, extensions: ms-vscode.node-debug, ms-vscode.node-debug2	8 files / 0.012MB
source-map	extensions: ms-vscode.node-debug, ms-vscode.node-debug2	18 files / 0.541MB
vscode-debugadapter	extensions: ms-vscode.node-debug, ms-vscode.node-debug2	18 files / 0.167MB
vscode-debugprotocol	node_modules/, extensions: ms-vscode.node-debug, ms-vscode.node-debug2	10 files / 0.304MB
semver	node_modules/, extensions/typescript	20 files / 0.158MB
-	-	-
Total:		1214 files / 7.162MB
Percent of extensions/ + node_modules/:		21% of files / 14% of size

Source codes inside VSCode:

Module directory name	How many files/size we can save
node_modules/nan/*.h	18 files / 0.190MB
node_modules/node-pty/deps/*.cc, *h	174 files / 0.815MB
node_modules/node-pty/lib/src/	24 files / 0.044MB
node_modules/xterm/src/	44 files / 0.340MB
-	-
Total:	260 files / 1.389MB

Development examples inside VSCode:

Module directory name	How many files/size we can save
node_modules/node-pty/lib/examples/	69 files / 0.276MB

With already reduced size of lodash ( Axosoft/nsfw#23 ) you can reduce size of next build on 9+MB and 2500+files! => Increase speed of installation on 41%! (currently main reason of slow installation - count of files)

[joaomoreno]

cc @Microsoft/vscode

[jrieken]

I have tried this: #15981. The ideas was to find extensions that depend on the same node_modules (same id and version) and to pull those one level up to have them shared. Risk was that node_modules might have static state which isn't meant to be shared, eg. the app insight module has a static singleton that cannot be used by different clients. So, the challenge is to know what node_modules can be safely shared across different clients.

[Tyriar]

nan | node_modules/, nsfw, node-pty

I created microsoft/node-pty#117 which might make the situation a little better by relying on the same version as nsfw.

It doesn't solve the issue of the other version being the one in node_modules though. It's a tricky situation as it's very dangerous to depend on a different version of a module than a module specifies. Trying to improve the install experience in other ways might be better than trying to squeeze node_modules as much as possible potentially at the cost of stability.

node_modules/node-pty/lib/src/

I believe this was due to a bad publish, created microsoft/node-pty#118 to prevent this in the future.

node_modules/node-pty/deps/*.cc, *h
node_modules/node-pty/lib/examples/

• These should be cleaned

node_modules/xterm/src/

Sourcemaps rely on this, it's not much overhead for really nice debugging of issues.

[Tyriar]

Related #30640

[joaomoreno]

To keep the number of issues in our inbox on a manageable level, we're closing issues that have been on the backlog for a long time but haven't gained traction: We look at the number of votes the issue has received and the number of duplicates issues filed.

Thanks for your understanding. Happy coding!

[SailorMax]

@joaomoreno Can you confirm, this closing was automatically or you do not interested in minimizing size and installation speed of VSCode?

[joaomoreno]

It was neither automatic nor am I uninterested. The issue was closed simply because we have other ones open. It's just not feasible to carry around 5000 issues, so hard choices need to be made.

[joaomoreno]

Also: #38825

[SailorMax]

Now, after you mention #38825 - I agree.
Did not saw it.

[joaomoreno]

No worries! Thanks for your passion! 👍