Add support for woff2 fonts

[bramp]

Currently woff2 fonts return "unsupported font format". Please support woff2.

[dmitshur]

Hi @bramp.

What's your timeframe for wanting this? Are you looking for a solution ASAP, or is this more of a general long-term feature request?

This happens to be something I've started working on recently, and I have a working prototype. It's still in early stages, but as of yesterday, it's successfully parsing a Go font woff2 file that I've been testing with.

I want to make further progress before pushing the code, and when it's done, I was considering sending a PR here (after finding out whether this repository is still actively maintained or not). If you want it sooner @bramp, I can share unfinished code.

[bramp]

I'm in no rush, but I came across this because I wanted to use the library to extract the Features the OpenType font support (such as Ligatures, Lining Figures, Small Capitals, etc).

I've actually spent my afternoon adding parsing of the features from the font (a PR will be coming shortly).

However, always happy to review unfinished code, if you want another pair of eyes on it.

[ConradIrwin]

Awesome! I’d love the feature extraction code. Woff2 is a nice to have, I was waiting for someone else to port the compression algorithm over, as it uses a relatively obscure scheme there was no go library at the time. P.S. Superhuman is hiring iOS engineers — k now anyone? $1947 referral bonus. Learn more ( https://superhuman.com/jobs ).  Sent via Superhuman iOS ( https://sprh.mn/[email protected] )

…

On Sat, Jan 20 2018 at 11:51 PM, Bhavesh Kakadiya < ***@***.*** > wrote: I'm in no rush, but I came across this because I wanted to use the library to extract the Features the OpenType font support (such as Ligatures, Lining Figures, Small Capitals, etc). I've actually spent my afternoon adding parsing of the features from the font (a PR will be coming shortly). However, always happy to review unfinished code, if you want another pair of eyes on it. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub ( #1 (comment) ) , or mute the thread ( https://github.com/notifications/unsubscribe-auth/AAFwQBRQv-Sl9HovHoJ2MEXNhg0tJH_9ks5tMux2gaJpZM4RloNf ).

[ConradIrwin]

Et voila: https://github.com/dsnet/compress

[dmitshur]

However, always happy to review unfinished code, if you want another pair of eyes on it.

Great, thanks! I will clean it up myself first, and send it for review when it's ready.

Et voila: https://github.com/dsnet/compress

Yep, github.com/dsnet/compress/brotli is exactly what I used as the Brotli decoder in my woff2 parser. There's no (pure Go) Brotli encoder yet, the status of that is being tracked at dsnet/compress#42. (A Brotli encoder is needed to implement a woff2 encoder.)

[dmitshur]

I have an update. I've cleaned up the code and it's ready for review. The implemented functionality so far is enough to parse all Go font family woff2 files that I was able to throw at it. It's definitely incomplete and not 100% spec compliant regarding validation of errors, etc. But one step at a time; it should be enough to resolve this issue. I wanted to get it to this point first and work on the rest in future changes.

However, I'll be transparent, one of the reasons I'm working on the woff2 package is because I want to use it as a pet project to test out a custom code review tool for Go that I'm (slowly) working on. To that end, I plan to self-host this woff2 decoder package (similar to https://dmitri.shuralyov.com/kebabcase) and do development there. In other words, I know it'd be easier to just put it on GitHub and use pull requests, but that's not my goal with this project.

Another factor is that it might make sense for this library/repository to just contain the woff2-related code inside. There's some stuff around tags that can be reused. It'd be easier to be able to change all relevant code with 1 PR rather than having send changes 2 separate repos.

But I do like the idea of having a dedicated woff2 decoder (and in the future, encoder) package, similar to how there are standalone packages for various image formats, such as image/png, image/jpeg, golang.org/x/image/bmp, etc.

So, how we proceed is largely up to you @ConradIrwin and @bramp:

1. If you're okay with having the woff2 decoder be a standalone package elsewhere and have sfnt be a client of that package, and deal with my indieweb-inspired experiment of developing a Go package not on github, I can send both things for review.

   If we go this route, I'm happy to help maintain things, and if there need to be any breaking API changes, I'll follow up here to keep things building (see my policy on breaking API changes here).

   Here's a preview of what the woff2 package API looks like so far: https://redpen.io/rk9a75c358f45654a8.

   (I also think this way would be more fun, but maybe it’s just me.)

2. Otherwise, if you wish, you're welcome to copy/use my woff2 decoder code and implement a copy of it inside this repo to resolve this issue. I'm licensing my code liberally with the Go license, but you can just reuse the existing MIT license of this repo. But I'll continue development of my standalone woff2 package separately.

[ConradIrwin]

@shurcooL hey! Absolutely fine if you want to host the woff2 decoder — we should probably start tagging dependency versions ourselves.

What would the diff look like to read woff2 files?

If I remember rightly, woff2 files are just a different container for sfnt, including some crazy compression rules. Does it makes sense to do woff2 -> sfnt and then re-use the sfnt parser in this library, or would it make sense to define a very basic common interface so that we can re-use the table parsing logic from this library, but rely on woff2 to extract and decompress each table?

type FontContainer interface{
   TableList() ([][4]byte, err)
   Table(tab [4]byte) ([]byte, err)
}

[dmitshur]

hey! Absolutely fine if you want to host the woff2 decoder — we should probably start tagging dependency versions ourselves.

What would the diff look like to read woff2 files?

Great! Let me send a PR to this repo to give you a better idea of what it'll look like, and you can review that. (Edit: Sent PR #14.)

If I remember rightly, woff2 files are just a different container for sfnt, including some crazy compression rules.

Yes, woff2 is very similar to sfnt and woff1. The compression rules are quite simple, they basically replace zlib with Brotli, that's about it. However, there are quite a few rules in https://www.w3.org/TR/WOFF2/ about what counts as invalid woff2 files that a spec-compliant parser must reject, e.g.:

The CompressedFontData stream MUST be compressed using the Brotli compression algorithm [Brotli]. If the decompression of the data block fails for any reason, the WOFF2 file is invalid and MUST NOT be loaded.

If the transformLength of the transformed loca table is not equal to zero, or if the encoded origLength does not match the calculated size defined above the decoder MUST reject the WOFF2 file as invalid.

My goal with the woff2 package is to implement all those for rules for correctness, and have the StrictParse-like behavior.

Does it makes sense to do woff2 -> sfnt and then re-use the sfnt parser in this library, or would it make sense to define a very basic common interface so that we can re-use the table parsing logic from this library, but rely on woff2 to extract and decompress each table?

Yes, that is the general direction I wish to follow. Inspiration for me is the image.Image interface that allows all image format encoder/decoders to able to work interchangeably on image data. However, I think we should take our time getting there, and take small, testable steps that get us closer, because figuring out a good common interface isn't trivial.

A good first step IMO is to resolve this issue—to add support for parsing woff2 files to font/sfnt package, and add tests for it. That'll allow all the commands in cmd/font to operate on .woff2 files, which is useful. Then, we can think about what priorities to accomplish next and work on them one at a time.