Adding support for procedural programs to drasil-code

[B-rando1]

As mentioned in #3868, once we move the Julia renderer over from OOProg to ProcProg, we lose the ability to generate Julia code because drasil-code only knows how to generate OOProg code. We need to enable drasil-code to generate procedural programs.

I don't think the 'main part' of the implementation will be too bad - a lot of things in drasil-code that currently require OOProg can be relaxed to SharedProg, which means that all renderers can use them.

I have two things where I'm less sure, though:

• How do we want to organize the distinction between OOProg and SharedProg? The example I'm currently looking at is genCode, the root function for generating code. Specifically this part:
  

  Drasil/code/drasil-gen/lib/Language/Drasil/Generate.hs

  Lines 151 to 158 in c86e117

  	let genLangCode Java = genCall Java unJC unJP
  	genLangCode Python = genCall Python unPC unPP
  	genLangCode CSharp = genCall CSharp unCSC unCSP
  	genLangCode Cpp = genCall Cpp unCPPC unCPPP
  	genLangCode Swift = genCall Swift unSC unSP
  	genLangCode Julia = genCall Julia unJLC unJLP
  	genCall lng unProgRepr unPackRepr = generateCode lng unProgRepr
  	unPackRepr $ generator lng (showGregorian $ utctDay time) sampData chs spec

  
  This works in the main branch because unJLC is of the same type as unJC and the rest, but in Switched the Julia renderer to ProcProg #3868 and beyond it's of a different type.
  • Would it make sense to have this function, and any others in a similar situation, take an Either? This would allow us to pass it Right JC or Left JLC, and let it go from there.
• How do we want to enforce the types of Choices that work with Julia?
  • For now, we could just throw errors 'on the fly', whenever the renderer tries to generate code that it can't.
  • It would be better to catch these sooner. We could do this either by analyzing Choices before starting, or by somehow constraining Choices to only allow certain options for certain types of output languages.

---

Edit: I just realized that at least for this function, we don't need Either. We can split up the genCall function into genCallOO and genCallProc to handle the input type signatures, and do the same for generateCode. Since genCall and generateCode simply return IO (), we don't need to worry about differences in output types.

[B-rando1]

I did some poking around in drasil-code. There are a total of 99 functions that required an instance of OOProg. Of those, only 18 of those constraints could be relaxed to SharedProg. This isn't what I had hoped for, as it means at least 81 functions will need to be split up in order to support ProcProg in drasil-code.

I'll take a stab at splitting a few up, and see how it goes.

[JacquesCarette]

I think splitting the genCall function (and generateCode) is a better idea.

I agree that we should check for valid Choices earlier. But that should be a subsequent PR, this one shouldn't be blocked by that.

My gut feel is that close inspection of those 81 functions will yield opportunities for improvements that will result in more things being able to go into SharedProg.

[B-rando1]

I made the necessary changes to drasil-code to have it generating Julia code again, which you can see in #3868. I didn't find any more functions that can 'naturally' be relaxed to SharedProg. The issue is that they all use things like mkVal, mkVar, convTypeOO, convExpr, etc. that all have a chance of giving an object back.

For some of the functions that call these, we can probably assume that when they call one of these functions they won't get an object back, but these assumptions aren't enforced by the underlying types. If we want to merge more of these functions, we need to either:

• Leave the underlying types alone, and investigate each function on a case-by-case basis. If we can be reasonably sure that in a certain case a function call won't return anything OO-related, we can switch the call to a SharedProg version.
• Investigate the types, and separate them into their shared, OO, and (eventually) procedural components. This will make the types harder to organize, but will hopefully make it easier to merge functions.

[JacquesCarette]

Definitely the second option is the way to go. I'd start with the leaf-most types and work my way up. This may well be best done in a dozen PRs, some of which may feel like they barely do anything -- but each will get incrementally closer.

[B-rando1]

I've been avoiding this for a while, but I figured I should try to take it a bit further if I can. I wasn't sure how to start, so I did another pass over the functions in question.

It looks like there are around 60 problematic functions left. I noted the reasons why each one is problematic. If you're interested I'll include them here.

Complete list of functions, with reasons for separation

FunctionCalls.hs:

• genAllInputCalls: relies on genInputCall, genDerivedCall, and genConstraintCall.
• genDerivedCall: relies on inOutCall
• genConstraintCall: relies on genFuncCall
• genCalcCall: relies on genFuncCall
• genOutputCall: relies on genFuncCall
• genFuncCall: relies on mkVal and fApp
• genInOutCall: relies on mkVar and fAppInOut Import.hs:
• value: relies on variable and convExpr
• variable:
  • Relies on inputVariable and constVariable
  • OO uses staticConst but Proc uses constant
• inputVariable: May return an object if inputs are bundled
• constVariable: May return an object if constants are bundled
• mkVal:
  • Relies on value
  • If the CodeVarChunk has an object, it returns an object
• mkVar:
  • Relies on variable
  • If the CodeVarChunk has an object, it returns an object
• genModDef:
  • Relies on genModuleWithImports, genFunc, and publicFunc
  • Different types of Module
• mkParam: relies on mkVar
• publicFunc: relies on genMethod
• privateMethod/privateFunction:
  • Relies on genMethod
  • For OO it's a method; for Proc it's a function
• genMethod: relies on mkVar and mkParam
• genFunc:
  • Relies on convStmt, mkVar, and genDataFunc
  • Only OO can do constructors
• genModFuncs: relies on genFunc and publicFunc
• readData: relies on mkVal, mkVar, and getEntryVarLogs
• getEntryVars: relies on mkVar, variable, and convType
• getEntryVarLogs: relies on getEntryVars
• convExpr:
  • Relies on convCall, and fApp
  • Only OO can do constructors calls, method calls, and object field access.
• convCall: relies on convExpr and mkVar
• convStmt: relies on convExpr and mkVar
• genDataFunc: relies on readData and publicFunc
• publicInOutFunc: relies on genInOutFunc
• privateInOutMethod/privateInOutFunc:
  • Relies on genInOutFunc
  • For OO it's a method; for Proc it's a function
• genInOutFunc: relies on mkVar GenerateGOOL.hs:
• genModuleWithImports: modules are different types and have different parameters
• genModule: relies on genModuleWithImports
• fApp: Only OO does method calls
• fAppInOut: Only OO does method calls Modules.hs:
• genMain:
  • Relies on genModule and genMainFunc
  • Different kinds of File
• genMainFunc: relies on mkVar, initConsts, getInputDecl, genAllInputCalls, genCalcCall, and genOutputCall
• initConsts:
  • Relies on mkVar and convExpr
  • OO returns an object in constants are bundled
• genInputMod:
  • Relies on genModule, genInputFormat, genInputDerived, and genInputConstraint
  • Different types of File
• getInputDecl:
  • Relies on mkVar
  • OO returns an object if inputs are bundled
• genCalcMod:
  • Relies on genModuleWithImports and genCalcFunc
  • Different types of File
• genCalcFunc: relies on mkVar, genCalcBlock, convStmt, and publicFunc
• genCalcBlock: relies on genCaseBlock, mkVar, and convExpr
• genCaseBlock: relies on convExpr and genCalcBlock
• genInputFormat: relies on publicInOutFunc, privateInOutMethod/privateInOutFunc, and readData
• genInputDerived: relies on publicInOutFunc, privateInOutMethod/privateInOutFunc, and genCalcBlock
• genInputConstraints: relies on publicFunc, privateMethod/privateFunc, sfwrCBody, and physCBody
• sfwrCBody: relies on chooseCstr
• physCBody: relies on chooseCstr
• chooseCstr: relies on convExpr, constrWarn, and constrExc
• constrWarn: relies on constraintViolatedMsg
• constrExc: relies on constraintViolatedMsg
• constraintViolatedMsg: relies on printConstraint and mkVal
• printConstraint: relies on convExpr
• genOutputMod:
  • Relies on genModule and genOutputFormat
  • Different types of File
• genOutputFormat: relies on mkVal and publicFunc Generator.hs:
• generateCode:
  • Relies on genPackage
  • Different types of Program
• genPackage:
  • Relies on genProgram
  • Different types of Program
• genProgram:
  • Relies on chooseModules
  • Different types of Program
• chooseModules:
  • Relies on genUnmodular and genModules
  • Different types of File
• genUnmodular:
  • Relies on genModuleWithImports, genMainFunc, genCalcFunc, genModFuncs, genInputFormat, genInputDerived, genInputConstraints, and genOutputFormat
  • Different types of File
• genModules:
  • Relies on genMain, genInputMod, checkConstClass, genCalcMod, genOutputMod, and genModDef

Summary:

These are the 'root causes' that are stuck not because of GOOL/GProc-related types, but because of drasil-code/drasil-lang reasons.

• variable
  • If the variable is a constant and constant representation is Const, then for GOOL it's a staticVar and for GProc it's a constant. We might be able to make both use constant, I'm not sure.
• inputVariable
  • If inputs are bundled, for GOOL it's an object and for GProc it's an error
• constVariable
  • If constants are bundled, for GOOL it's an object and for GProc it's an error
• mkVal
  • CodeVarChunks have two fields: a CodeChunk for the variable and a Maybe CodeChunk for an object holding the variable. GProc can't do objects.
  • codeType can also return Object because Actors are mapped to Object, so GOOL uses convTypeOO and GProc uses convType.
• mkVar
  • CodeVarChunks have two fields: a CodeChunk for the variable and a Maybe CodeChunk for an object holding the variable. GProc can't do objects.
• genFunc
  • The Func type can either be a function or a constructor. GProc can't do constructors.
• convExpr
  • CodeExpr has the New, Message, and Field constructors, each of which has to do with objects, which GProc can't do.
• fApp
  • With GOOL, a function may be a private method of the current object, so there's an extra case for a self method call.
• fAppInOut
  • With GOOL, a function may be a private method of the current object, so there's an extra case for a self method call.
• initConsts
  • If constants are bundled, GOOL declares a new object but GProc throws an error
• getInputDecl
  • If inputs are bundled, GOOL declares a new object but GProc throws an error

Analysis:

There are a few common problems:

• Bundling: GProc can't do bundling, and if it could it would use structs instead of objects.
• CodeVarChunks implicitly support the idea of objects with their Maybe CodeChunk field.
• Spaces implicitly support the idea of objects with their Actor value.
• The Func type from the Mod AST includes a CtorDef value for constructors.
• CodeExpr has fields for accessing constructors, methods, and members of a class/object.
• I'm not exactly sure what's going on with fApp, but it has a possibility of calling a method on self.

Basically, these are all legitimate reasons that not only do the implementations of these functions need to be different, their return types (and those of any functions that use them) need to be different. I'm not really sure what we can do about that.

• We could try splitting things up a little bit - e.g. split up CodeVarChunk into two data types so that one of them can be safely shared. I'm just not sure how effective that would actually be in most of these cases. There might be more digging that needs to be done here.
• I'm just thinking off the top of my head, but it seems to me that GOOL was so easy to split up because of its tagless approach, or at least because it uses typeclasses. Would we be able to get the polymorphism we need by doing something along the same lines here?

[JacquesCarette]

Excellent analysis. Clearly too late to do anything about this right now.

Your last point is exactly right: we should be able to use typeclasses to get extra polymorphism to be able to deal with this. The second part of the solution would be to make a number of the above functions more 'semantic' and less code- or feature-centric. Your example of variable / constant goes in this direction.

Basically, we'd want to make sure we represent "ideas" in Drasil (especially in drasil-lang and drasil-code) and let the backend decide how to turn that into working code for each language.