Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SSL #8

Open
DanGrayson opened this issue Jan 5, 2020 · 59 comments
Open

SSL #8

DanGrayson opened this issue Jan 5, 2020 · 59 comments
Assignees

Comments

@DanGrayson
Copy link
Member

We should switch to SSL-only service for increased security for our users.

@DanGrayson DanGrayson self-assigned this Jan 5, 2020
@DanGrayson
Copy link
Member Author

We already are on https, by math department policy, so the only non-SSL site is our mirror.

@mahrud
Copy link
Member

mahrud commented Apr 19, 2020

Two ways to have https for the mirror:

  • Cloudflare provides free tls certificates (near zero setup)
  • https://letsencrypt.org provides is a free certificate authority with automatic tooling for updating the certificate.

@DanGrayson DanGrayson reopened this Apr 19, 2020
@DanGrayson DanGrayson removed their assignment Apr 19, 2020
@mahrud
Copy link
Member

mahrud commented Aug 12, 2020

Why is this under discussion? Currently https://macaulay2.com/ doesn't work, and two potential solutions was provided.

@mahrud
Copy link
Member

mahrud commented Jan 12, 2021

@DanGrayson None of the links in your emails today work for me, as they all use https. The fix is nearly trivial! Do you need help setting it up?

[mahrud@noether ~]$ curl https://macaulay2.com
curl: (28) Failed to connect to macaulay2.com port 443: Connection timed out

@DanGrayson
Copy link
Member Author

Just try this instead:

$ curl macaulay2.com
<html><body>You are now redirected to<br><a href="https://www.math.illinois.edu/Macaulay2/">https://www.math.illinois.edu/Macaulay2/</a></body></html>

@mahrud
Copy link
Member

mahrud commented Jan 12, 2021

This is trivial to fix and would cost $0, if anybody cared.

@mikestillman is it okay with you that the links in the emails don't work, and the offered solution is to "remove the 's'"?

@DanGrayson
Copy link
Member Author

I finally figured out which "emails" you're referring to. I'll see what I can do.

@DanGrayson
Copy link
Member Author

Joker.com doesn't seem to offer https url forwarding (see https://joker.com/faq/content/52/111/en/what-is-url-forwarding-web-forwarding.html ). I'll send out a correction.

@mahrud
Copy link
Member

mahrud commented Jan 13, 2021

...

You can switch to using Cloudflare as the name service, which only charges you the amount that they need to pay Joker for registrar fees. Here is the doc: https://developers.cloudflare.com/registrar/

Then proxy the traffic through them. There, you can enable TLS for free. They will provide the certificate and take care of renewals. This also reduces load on the server.

I have worked for the company and know how their website works. Would you like me to do it for you?

All of this costs nothing extra. You can also do it without Cloudflare through Let's Encrypt, but the config is more involved.

The fact that this issue has been open for years is really silly.

@DanGrayson
Copy link
Member Author

DanGrayson commented Jan 13, 2021

Yes, but that doesn't address the issue at all, which is that our mirror (on one of my virtual machines) serves up http.

Re: "The fact that this issue has been open for years is really silly."

It's been 1 year.

@mahrud
Copy link
Member

mahrud commented Jan 13, 2021

Yes, but that doesn't address the issue at all, which is that our mirror (on one of my virtual machines) serves up http.

Dan, I don't think you understand what Cloudflare does. Cloudflare sits between your server and the internet. It serves https to the world, and queries your server for requests that are not cached. If a request is cached, it'll show the result from it's servers across the globe. This will significantly reduce load on your server, and provide https for users.

Would you like to call and do it together?

@mahrud
Copy link
Member

mahrud commented Jan 13, 2021

To reiterate: your server doesn't need to serve https, http is enough. Cloudflare's servers have https servers running and serving the content that they get through http from your server.

@DanGrayson
Copy link
Member Author

Oh! Then it's worth trying.

Either that, or we could just get rid of the mirror. Our load is rather light, I'm sure.

@DanGrayson
Copy link
Member Author

On second thought, if the data passes insecurely from cloudflare to our server, then the connection is insecure, so there doesn't seem to be any point in doing it.

@mahrud
Copy link
Member

mahrud commented Jan 13, 2021 via email

@DanGrayson
Copy link
Member Author

DanGrayson commented Jan 13, 2021

Re: "the problem right now is that https://macaulay2.com doesn't work"

That's no longer a problem -- I sent out a correction giving the correct URLs.

@mahrud
Copy link
Member

mahrud commented Jan 13, 2021 via email

@mahrud
Copy link
Member

mahrud commented Jan 13, 2021

At least try it for a while, if you don't notice a significant raise in how fast the website loads, then revert it.

@DanGrayson
Copy link
Member Author

At least try it for a while, if you don't notice a significant raise in how fast the website loads, then revert it.

I won't notice any speed increase, as I'm 3 miles from macaulay2.com.

@mahrud
Copy link
Member

mahrud commented Jan 13, 2021 via email

@dimpase
Copy link

dimpase commented Jan 13, 2021

Cloudflare's sales pitch strikes be as biased (oh well, surprise).

Setting up letsencrypt does not require changing internet providers, and does not tie you up to yet another dominant Internet player. I'd do letsencrypt.

@DanGrayson
Copy link
Member Author

Is "letsencrypt"'s method like cloudflare's method, in that the portion of the communication channel between letsencrypt and our server is unsecure?

@dimpase
Copy link

dimpase commented Jan 13, 2021

No. letsencrypt issues you a normal SSL certificate, which needs to be updated every month or so.
Clients to do this renewal are available for every major OS.

@dimpase
Copy link

dimpase commented Jan 13, 2021

letsencrypt is a true open-source solution, too.

@DanGrayson
Copy link
Member Author

PS: Could we forget about virtual machines and do everything through github? If so, how would that actually work?

@dimpase
Copy link

dimpase commented Jan 15, 2021

you can definitely host a website on GitHub Pages, one that would be quite sufficient for M2 purposes, and
do binary releases using GitHub's releases (one can "edit" the release and upload whatever "binary assets" you want - there are some limits on file sizes, but IMHO M2 is quite OK in this regard).

@DanGrayson
Copy link
Member Author

You're right - I was able to upload all my binary release files to the github release.

@DanGrayson
Copy link
Member Author

Is there a limit for github pages total size?

@dimpase
Copy link

dimpase commented Jan 15, 2021

1 Gb for GH Pages

Needless to say, it's the website only, source code and releases of the project don't count.

@mahrud
Copy link
Member

mahrud commented Jan 15, 2021

Also they're static, so the "random paper referring to Macaulay2" widget wouldn't work.

@DanGrayson
Copy link
Member Author

@mahrud
Copy link
Member

mahrud commented Jan 15, 2021

I don't know what you mean. Typically github pages are created using markdown and populated using Jekyll, rather than uploading html files. This was partially the goal of the markdown command that I wrote, so you could make package documentation like this: https://mahrud.github.io/m2doc/packages/FGLM/ (this is a work in progress)

@DanGrayson
Copy link
Member Author

Oh, now it started working. I guess there was a sizable delay.

@dimpase
Copy link

dimpase commented Jan 15, 2021

"random paper referring to Macaulay2" widget wouldn't work.

IMHO Pages do allow embedded javascript (perhaps curated somehow). So YMMV.

@DanGrayson
Copy link
Member Author

1 Gb for GH Pages

Needless to say, it's the website only, source code and releases of the project don't count.

I think the repository might be 0.5gb if we include the documentation for 2 versions. But then after a new version is
released, we need a way to prune the older documentation from the repository, so we don't exceed our limit.

@dimpase
Copy link

dimpase commented Jan 16, 2021

one can split the site into parts, e.g. create a dedicated GH Pages site for packages, a dedicated site for docs, etc.

@DanGrayson
Copy link
Member Author

Splitting it up doesn't reduce the total size.

@DanGrayson
Copy link
Member Author

DanGrayson commented Jan 16, 2021

Here's a procedure for removing all the history from a repository, but keeping the files:

    git checkout --orphan new-master
    git branch -D master
    git branch --move master
    git reflog expire --all --expire=now
    git gc --prune=now --aggressive
    git add -A
    git commit -m 'add all the files'

@dimpase
Copy link

dimpase commented Jan 16, 2021

Splitting it up doesn't reduce the total size.

You can create, say, github.com/Macaulay2-packages
and associate a separate GH Pages with it, for packages info.
This won't count into your 1Gb quota for github.com/Macaulay2

@DanGrayson
Copy link
Member Author

Well, eventually that will fill up, too, if we keep adding documentation html to it.

@dimpase
Copy link

dimpase commented Jan 16, 2021 via email

@DanGrayson
Copy link
Member Author

I've just checked - the html documentation for 1.17 is 125mb. So 8 of those will fill it. 4 years.

@dimpase
Copy link

dimpase commented Jan 16, 2021

I wouldn't worry about the docs for more than 2 releases at the same time.

@dimpase
Copy link

dimpase commented Jan 16, 2021

By the way, Sagemath's html docs are 23Mb, it's strange that M2's docs, smaller by pagecount, IMHO, take so much more space.

@DanGrayson
Copy link
Member Author

Actually, if I remove the example output files, it's just 93mb, with 9000 files. So 10K per html file on average. We have lot of html that's automatically generated -- that could explain some of the size.

@mahrud
Copy link
Member

mahrud commented Jan 17, 2021

Alternatively, we can use the markdown version of the documentation. With markdown, the template is only written in a single file, so we save 9000*[the size of our html header section]. This is a benefit of jekyll. See http://jekyllbootstrap.com/lessons/jekyll-introduction.html.

@DanGrayson
Copy link
Member Author

Alternatively, we can use the markdown version of the documentation. With markdown, the template is only written in a single file, so we save 9000*[the size of our html header section]. This is a benefit of jekyll. See http://jekyllbootstrap.com/lessons/jekyll-introduction.html.

I don't follow the logic here, since the html files have to be static and have to be generated, but as an experiment, it might be worthwhile generating all the html files your way, to see if it's less than 93mb, and to show us the result.

@dimpase
Copy link

dimpase commented Jan 17, 2021 via email

@DanGrayson
Copy link
Member Author

Yes, here are some examples:

          <li>
<span><tt>&quot;addHook(...,Strategy=>...)&quot;</tt> -- see <span><a title="add a hook function to an object for later processing" href="../../Macaulay2Doc/html/_add__Hook.html">addHook</a> -- add a hook function to an object for later processing</span></span>          </li>
          <li>
<span><tt>&quot;annihilator(...,Strategy=>...)&quot;</tt> -- see <span><a title="the annihilator ideal" href="../../Saturation/html/_annihilator.html">annihilator</a> -- the annihilator ideal</span></span>          </li>
          <li>
<span><tt>&quot;associatedPrimes(...,Strategy=>...)&quot;</tt> -- see <span><a title="find associated primes" href="../../PrimaryDecomposition/html/_associated__Primes.html">associatedPrimes</a> -- find associated primes</span></span>          </li>
          <li>
<span><tt>&quot;mingens(...,Strategy=>...)&quot;</tt> -- see <span><a title="a Strategy option value" href="../../Macaulay2Doc/html/___Complement.html">Complement</a> -- a Strategy option value</span></span>          </li>

@mahrud
Copy link
Member

mahrud commented Jan 17, 2021

I don't follow the logic here, since the html files have to be static and have to be generated, but as an experiment, it might be worthwhile generating all the html files your way, to see if it's less than 93mb, and to show us the result.

With jekyll, github generates the html files, not us. The repository contains only markdown files plus a few template files written in html.

@mahrud
Copy link
Member

mahrud commented Jan 17, 2021

Regardless, this is a bigger project that probably won't happen for some time. We've collectively provided multiple ways to get TLS to work, all of which are improvements over the current situation.

@DanGrayson
Copy link
Member Author

Oh, okay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants