Design of procedural languages in GOOL

[B-rando1]

This discussion started in #3789. The summary is, I was bringing up an issue I was having with translating GOOL classes into Julia libraries, and @smiths brought up the question of whether this is the right way to go. The danger of translating OO patterns into a non-OO language is that we might end up with code that is trying to emulate the OO patterns rather than achieving the functionality in a way that is idiomatic in that language.

As I mentioned in the issue, I don't think this is necessarily the case, at the very least not for C code. The examples I have seen of libraries in procedural languages usually end up looking quite similar to classes anyway, so I don't think this is a bad way to go.

With that said, I might need to do a bit more research on Julia (and potentially Matlab as well) to see if there different patterns that it considers idiomatic, in particular with respect to its more unique features. The documentation certainly hints at possible patterns, though as of yet I am unsure of whether these patterns are used in production code. I will do some digging, and I might also write an example 'by hand' if I think that will be helpful.

An example I remembered of a procedural library deviating from the OO way of doing things is GameMaker's particle system. In GameMaker to create particles you need a particle system, a particle type, and a particle emitter. These are separate constructs in GameMaker, so functions often take an instance of each (e.g. part_emitter_burst needs one of each, as well as the number of particles to generate). In OO this would likely be done differently, for example with has-a relationships. Using structs we can still express has-a relationships in procedural languages, but this might serve as evidence that that isn't always the most idiomatic way of doing it.

I guess that my question is, how fragmented do we want to allow GOOL to become? Currently we have a nice system where we generate one GOOL program, and it cleanly translates to all of the targets (with a few minor exceptions). Do we really want to push the differences between languages above GOOL and into drasil-code? It seems to me that GOOL would be neither Generic nor OO at that point 😄. Depending on what I find about Julia and Matlab it might not come to that, but I think it's an important question that we should have an answer ready for.

Some of this is just my thoughts, so what are your thoughts @smiths @JacquesCarette @balacij? Do you agree that it's still worthwhile to translate GOOL classes into Julia libraries? Do you think we should investigate generating GOOL code designed specifically for some targets, even at the risk of losing reusability?

[B-rando1]

I like this Turtle Graphics example in Julia: https://github.com/JuliaGraphics/Luxor.jl/blob/master/src/Turtle.jl.

Here you see it using multiple constructors. It doesn't look like they're chaining constructors (they all call the 'true' constructor directly) so this is something that would work in all of our current targets except Python.

Turtle(x::Float64, y::Float64) = Turtle(x, y, true, 0, (0, 0, 0))
Turtle(pos::Point=O) = Turtle(pos.x, pos.y, true, 0, (0, 0, 0))
Turtle(pos::Point, pendown::Bool) = Turtle(pos.x, pos.y, pendown, 0, (0, 0, 0))
Turtle(pos::Point, pendown::Bool, orientation::Real) = Turtle(pos.x, pos.y, pendown, orientation, (0, 0, 0))
Turtle(pos::Point, pendown::Bool, orientation::Real, col::NTuple{3, Number}) = Turtle(pos.x, pos.y, pendown, orientation, col)
Turtle(pos::Point, pendown::Bool, orientation::Real, r, g, b) = Turtle(pos.x, pos.y, pendown, orientation, (r, g, b))

Here you can see the type signature for one of its functions. All the other functions are very similar as well.

function Forward(t::Turtle, d=1)

I like how straightforward this library is compared to some others. It's very easy to read, and the API is simple and consistent. It also demonstrates that a traditional library structure can be used in Julia.

[JacquesCarette]

This seems like an interesting form of overloading.

[smiths]

I recently read an article about how Julia solves the Expression problem via multiple dispatch The multiple dispatch approach isn't really OO or functional. I need to see some more examples to really get it, but this approach does seem like something different.

From our past conversations, I believe @JacquesCarette is fine if GOOL isn't just for OO.

I think we should try to follow the approach we've been using of not deciding the "best" option, but instead supporting as many different patterns as we can. If a certain pattern isn't possible in a given language, then that code cannot be generated.

Since Julia will be able to generate code that is like our OO code, we should generate that code. In the future, when we understand multiple dispatch better, we could give the option of generating code that follows this paradigm. The generator in this case wouldn't work for languages that can't be made to support it.

GOOL can grow as new cases come up. I do think it will be advantageous at some point to find some idiomatic Julia code. Our current examples are too simple to highlight the differences between languages. It might help when we get indexing working, since Julia and Matlab are particularly well suited to processing vectors/matrices.

[B-rando1]

That was an interesting article @smiths! This is more of an aside, but it reminded me of Haskell's typeclasses. I noticed on the Wikipedia page for the Expression problem that the Tagless-final strategy is another solution to the expression problem, so they seem to be similar ideas, at least.

I think you might be right, maybe we should take a step back and focus on what's easiest to translate, and with more time to think about it and look at examples, we can find a better approach to these problems. It does seem like more of the work will have to be done by drasil-code than I initially thought.

[B-rando1]

I've been thinking about this some more. I think that article demonstrates something I hadn't thought of before: it would be beneficial for our code not just to be idiomatic in its own self-contained way, but to allow other programs to interop with it in an idiomatic way.

I don't know if anything has to change to achieve that level of interoperability (it might come 'for free' in Julia), but I think it's a good principle to keep in mind.

[B-rando1]

By the way, I started a wiki page about the design decisions related to converting GOOL classes into struct libraries. Here is the link. It's currently quite basic and somewhat out-of-date, but I'll do my best to update it.

[JacquesCarette]

Thanks for doing this.

[B-rando1]

Another place that I should have thought to mention earlier is the Julia standard library!
The List API is probably the best place to look.

• Everything under 'Basic functions' follows the 'OO style' of taking an instance of the data structure as its first argument.
• Some of the more complicated functions have a more unique signature, but those are much more specialized functions that don't even have counterparts in most list APIs, so I'm not sure that they're the best examples to go off of.
• The findfirst function is the only real exception I could find. In one of its implementations, it takes a predicate, then a list, and returns the index of the first element of the list that satisfies the predicate. I'm not really sure why they chose to put the predicate before the list, it seems like an odd choice to me.
• The API also makes heavy use of multiple dispatch. This is mainly for user convenience, e.g. the findfirst function has an implementation that only takes a list of Bool, and returns the index of the first element that is true.

[JacquesCarette]

First: I don't think we should be trying to 'translate' OO idioms to non-OO languages. It's not that it won't work -- it will work well enough for a while, and then get increasingly hacky, until it feels like it's not working anymore.

What we should be doing is figuring out how to "bundle functionality" together. This belongs in drasil-code. There may be a need for a GenProcLang (GPL is too overloaded, it doesn't bring to mind 'Generic Procedural Language') beside GOOL. They would share a huge amount (i.e. they are both imperative languages at their core).

The question then becomes: where do the OO assumptions come in (in the current code base)? Then we can figure out what to do about it.

[B-rando1]

That's a good point about the danger of doing too much in GOOL (as opposed to drasil-code), how it will start looking hacky if we take it too far. It sounds, then, like we want GOOL to translate GOOL code to code with the same meaning (as opposed to the 'equivalent but not the same' meaning I was attempting with translating objects to structs). Correct me if I misunderstood your comment.

From this, should I stop any work involving objects in procedural languages, and focus solely on translating things that have a 1-to-1 translation?

• If this is the way to go, then I would say we should probably add support for structs and/or hashmaps to GOOL at some point, as procedural languages would surely benefit from having some way of bundling data together. This doesn't have to be right away, but we should keep it in mind.

Then from there, we'll need to separate GOOL into the Venn diagram [OO [OO and Procedural] Procedural]. We might not need the Procedural-only part, but definitely will need the OO-only part if we want to benefit from Haskell's type system. There are a few structures in GOOL that assume a renderer implements all of the OO features, and we'll have to split that up somehow. I think this is in-line with what you meant when you said we may need a GenProcLang beside GOOL.

After that, we'll need to modify drasil-code to have more procedural-friendly options. We could think about having it generate multiple sets of GOOL code for its different targets, so that we don't have the problem we currently have that if your set of Choices doesn't work with a target language, you're out of luck. Alternatively we could have the user hand-pick multiple sets of Choices (as in the Projectile example), and generate some languages in one set, and other languages in others.

I think it's probably best (for me at least) to think about this for a few days, so I'll put it on the backburner and we can continue discussing in the meantime.

[JacquesCarette]

First paragraph: you understand correctly.

On your next question: I wouldn't quite say 1-to-1 translation. I would say meaning-preserving translation, and definitely only one-way (i.e. from our internal high-level representation to an actual PL).

In theory, structs are degenerate objects, i.e. those with no methods. In practice, it does make sense to have a separate representation for them so that we get this invariant "for free" because it is part of the representation itself. Adding hashmaps would indeed be useful.

I agree with the Venn diagram split.

As a fairly large design, it's probably wise to think about it, in the background, over several days.

[B-rando1]

I spent some time today tracing out the typeclasses in ClassInterface.hs, trying to get a grasp on how we can start splitting it up into OO-only, OO and Procedural, and Procedural-only typeclasses. I created a draw.io diagram mapping them out, viewable here.

I have two things that immediately arise from this:

• First is a question: what is StatePattern supposed to do? From the name it sounds like a design pattern, but it seems to be implemented as a macro for a switch statement. There's also no examples of it or test cases with it, so I'm not sure what to do with it.
• Second is an observation. The typeclasses seem to be more tightly coupled to OO than I had hoped. There are 11 typeclasses on the Procedural side of the diagram that directly mention OO terms, and most of the other typeclasses have a few of them as a dependency.
  • I don't think this is the end of the world, but it'll take some work. I'm hoping that I can start at the leaves of the dependency tree, splitting typeclasses into their OO and Procedural counterparts. I think with some luck we'll end up with a new tree where OO stuff only shows up in the OO section.

[JacquesCarette]

This "seems simple but spins out of control" is rather common. Glad you spotted it fairly early instead of letting it grow and grow.

Small PRs, some of which might even feel like sideways steps, will get you there.

Splitting things into files seems like a good idea. One can then even use all of them (by importing qualified) if/when needed.

[B-rando1]

I had a thought about name conflicts. Splitting things up into different files will be enough for drasil-gool, but then other folders access them through GOOL.Drasil, which bundles all of GOOL into one interface. Because of that, exported names (including the typeclasses we're talking about) still need to be unique, I believe.

Is there a way we could change how we export names that would avoid name conflicts? There are a good number of typeclasses and functions that will require prefixes in their names otherwise.

• We could split up the file GOOL.Drasil into 3 files, each one corresponding to OO, Proc, and Common, respectively.
• We could create two new directories for OO and Proc. I think we are planning to do this eventually, just we decided it would be better to get it working before we do too much refactoring.
• There might be other options that I don't know about.

What are your thoughts?

[JacquesCarette]

The two new directories seem to be the most future-proof.

[B-rando1]

Alright, sounds good. How much setup is involved to do that? E.g. editing the stack.yaml files, etc.

[JacquesCarette]

@balacij can help with the details.

[B-rando1]

I was looking at ScopeSym (the old one, that has to do with public vs private). It's used by function, inOutFunc, and docInOutFunc. I had assumed that ScopeSym should only be used with OO-features, but that doesn't actually seem to be the case.

From experimenting, it looks like the use of ScopeSym with functions, similar to with methods, defines whether a function is visible to other modules.

• In Java and C#, functions are implemented as static methods, so this translates in a very obvious way.
• In Swift, functions are declared at the top-level as in Python and C++, but if they are given the private keyword, they can only be called from within the module they are in.
• Python and C++ don't do anything with the scope given to functions.

Based on this, I believe my initial assessment was incorrect, and ScopeSym should continue to be common between OO and Procedural languages.

[B-rando1]

It seems like the main focus(es) of this discussion are finished. We chose a focus on how to deal with the differences between OO and procedural languages, and were able to make the necessary changes.

There's still a few loose ends, but #3871 and #3904 take care of them for the most part.