Dlls and ABI
Posted: March 30th, 2021, 4:10 am
I probably don't have time to explain everything I want to ask right now, but I'm having a hard time getting the right answers in my brain with how this should work and should be done.
I have so many questions on this. I want to make it correctly, but can't find the answers. It all seems very vague. At the very least it seems compromised.
Say I want to make a utility dll to put things in to use from my exe.
Reading I have done is not giving me 100% answers. Every option seems compromised.
To make this dll project and get the things I want to use available outside the dll, I found 3 main options.
1 - Decorate (my term) classes in the dll with __declspec(dllexport/import) (depending on #define flag in .cpp file) to export the class and import where I use / include it.
2 - Similar, but only decorating the class functions (I think) with extern "C" (I think) to export them as C functions.
3 - Create factory functions and a DEF file to export the function (export by ordinal / assigning a number to each function).
I'm not clear on this (hence this topic) but from my fried memory on it:
The compromises I found are:
1 - name mangling means dll project may need the projects that reference it to be built if it is changed even if what is being used is unchanged.
2 - with C functions, overloading is not possible. And you're not exporting the class, just the functions in it?
3 - Similar to C functions, exported functions must be unique throughout the entire dll project.
Benefits:
1 - Seems to be the neater, C++-ier way of doing it. Convenient.
2 - I'm not very clear on this as I felt it was too compromised between both 1 and 3 options.
3 - Has the benefit of being able to add exported functions without needing to rebuild referencing projects. This to me (coming from C#) seems like a great benefit. You can fix bugs and add features without fear of breaking (by needing to recompile) referencing projects. However, in my use, it is probably irrelevent, but I want it.
I want to be able to change and build the dll project as flexibly as possible. Meaning I don't want to have to rebuild referencing projects if the interfaces they deal with aren't changing. Having a setup whereby compiling doesn't break that would be great.
I want to retain C++ and not have to resort to C (no overloading, odd (to me) function signatures and relying on lots of free and factory functions).
I want to export interfaces (pure virtual ABCs) but don't like the idea too much of free factory functions to pump them out. Maybe I just need to get used to the idea and accept it is what it is, if that is the way.
Is this a pipe dream? How can I best do it? Don't get me wrong I've gotten stuff working, but I feel it's not being done properly. I've even bounced back and forth between two different ways of doing it.
I'm considering pimpl pure virtual ABCs now, however I don't quite understand the dll boundary yet, and don't want to spend hours trying things and finding the compromises near the end.
Memory management - what is the best way to do this? Factory methods that generate the objects in the dll and give you a pointer seem the way. A pimpl ABC is basically this where the implementation is hidden under a pointer.
Dll boundary - types:
I'm not getting what is safe to "pass" to a dll. Do I only need to worry about interchange of data in the exported factory functions? Once I get an object using a factory function, is it now OK that the interface of that object has all sorts of STL types as parameters?
As I understand the problem lies in different compilers, or compiler versions, or compiler options being used to compile the different module?/translation unit?/project? This can lead to the types not being structured the same, even ints (for example) not being the same size (one side might expect 16 bit and one 32).
Dll boundary - memory management:
Everything needs to be constructed and destructed on the same heap. Does it matter if an object constructed on the dll's heap is passed a pointer to something created on the exe (referencing project)'s heap? Or should this object (or it's creating dll) be constructing it's things only?
What does it mean to destroy something on another heap? Is calling delete inside an object created on one heap, on something created on another the definition of this? Is newing something from a referenced project bad or is it only bad if that is then deleted on another heap / from something created on another heap?
Some stackoverflow information points to linking to STL (? I think) by dll not statically on every project to ensure a common heap is used, to make sure there are no cross-heap memory problems. This seems like a bandaid; I'd rather do it right. Thoughts? I do like the idea of linking this way to not bake in the STL used into the binaries, but also it requires the user to have the MSVC++ redistributable installed. Thoughts on this?
I have looked into making COM objects and found an older tutorial that glossed over the more technical parts. It was interesting to see that the way I assumed you would make them is basically how it was done.
I watched a 2017 cppcon video on audio engines last night and found that the general design of them is very similar to how I have been developing mine, based on basic system / learnings from chili. Just mentioning these to, I dunno, give me a bit of a pat on the back for seeming like I'm on the right path to doing things right as I'm feeling quite overwhelmed with trying to get these dll projects right.
I hope someone can point me in the right direction. I feel chili probably has the best idea, but at the same time, I'm not sure this is something he's even looked at much. Plus time man, lol this is deep stuff.
I have so many questions on this. I want to make it correctly, but can't find the answers. It all seems very vague. At the very least it seems compromised.
Say I want to make a utility dll to put things in to use from my exe.
Reading I have done is not giving me 100% answers. Every option seems compromised.
To make this dll project and get the things I want to use available outside the dll, I found 3 main options.
1 - Decorate (my term) classes in the dll with __declspec(dllexport/import) (depending on #define flag in .cpp file) to export the class and import where I use / include it.
2 - Similar, but only decorating the class functions (I think) with extern "C" (I think) to export them as C functions.
3 - Create factory functions and a DEF file to export the function (export by ordinal / assigning a number to each function).
I'm not clear on this (hence this topic) but from my fried memory on it:
The compromises I found are:
1 - name mangling means dll project may need the projects that reference it to be built if it is changed even if what is being used is unchanged.
2 - with C functions, overloading is not possible. And you're not exporting the class, just the functions in it?
3 - Similar to C functions, exported functions must be unique throughout the entire dll project.
Benefits:
1 - Seems to be the neater, C++-ier way of doing it. Convenient.
2 - I'm not very clear on this as I felt it was too compromised between both 1 and 3 options.
3 - Has the benefit of being able to add exported functions without needing to rebuild referencing projects. This to me (coming from C#) seems like a great benefit. You can fix bugs and add features without fear of breaking (by needing to recompile) referencing projects. However, in my use, it is probably irrelevent, but I want it.
I want to be able to change and build the dll project as flexibly as possible. Meaning I don't want to have to rebuild referencing projects if the interfaces they deal with aren't changing. Having a setup whereby compiling doesn't break that would be great.
I want to retain C++ and not have to resort to C (no overloading, odd (to me) function signatures and relying on lots of free and factory functions).
I want to export interfaces (pure virtual ABCs) but don't like the idea too much of free factory functions to pump them out. Maybe I just need to get used to the idea and accept it is what it is, if that is the way.
Is this a pipe dream? How can I best do it? Don't get me wrong I've gotten stuff working, but I feel it's not being done properly. I've even bounced back and forth between two different ways of doing it.
I'm considering pimpl pure virtual ABCs now, however I don't quite understand the dll boundary yet, and don't want to spend hours trying things and finding the compromises near the end.
Memory management - what is the best way to do this? Factory methods that generate the objects in the dll and give you a pointer seem the way. A pimpl ABC is basically this where the implementation is hidden under a pointer.
Dll boundary - types:
I'm not getting what is safe to "pass" to a dll. Do I only need to worry about interchange of data in the exported factory functions? Once I get an object using a factory function, is it now OK that the interface of that object has all sorts of STL types as parameters?
As I understand the problem lies in different compilers, or compiler versions, or compiler options being used to compile the different module?/translation unit?/project? This can lead to the types not being structured the same, even ints (for example) not being the same size (one side might expect 16 bit and one 32).
Dll boundary - memory management:
Everything needs to be constructed and destructed on the same heap. Does it matter if an object constructed on the dll's heap is passed a pointer to something created on the exe (referencing project)'s heap? Or should this object (or it's creating dll) be constructing it's things only?
What does it mean to destroy something on another heap? Is calling delete inside an object created on one heap, on something created on another the definition of this? Is newing something from a referenced project bad or is it only bad if that is then deleted on another heap / from something created on another heap?
Some stackoverflow information points to linking to STL (? I think) by dll not statically on every project to ensure a common heap is used, to make sure there are no cross-heap memory problems. This seems like a bandaid; I'd rather do it right. Thoughts? I do like the idea of linking this way to not bake in the STL used into the binaries, but also it requires the user to have the MSVC++ redistributable installed. Thoughts on this?
I have looked into making COM objects and found an older tutorial that glossed over the more technical parts. It was interesting to see that the way I assumed you would make them is basically how it was done.
I watched a 2017 cppcon video on audio engines last night and found that the general design of them is very similar to how I have been developing mine, based on basic system / learnings from chili. Just mentioning these to, I dunno, give me a bit of a pat on the back for seeming like I'm on the right path to doing things right as I'm feeling quite overwhelmed with trying to get these dll projects right.
I hope someone can point me in the right direction. I feel chili probably has the best idea, but at the same time, I'm not sure this is something he's even looked at much. Plus time man, lol this is deep stuff.