The Mysterium Network project is built on Golang (Go), which is a statically compiled language and offers a rich standard library. Go is syntactically similar to C but comes out as the winner when it comes to memory safety, garbage collection, structural typing, and CSP style concurrency.
So how have we integrated the C++ OpenVPN 3 library into a Golang Mysterium Node? We are using OpenVPN and this was our first protocol, which was used as an external binary (executable file). This basically means that a Mysterium Node and OpenVPN are two different processes which communicate using OpenVPN config and IPC (local sockets to be exact).
Now, this has some limitations – for example, software distribution becomes complicated as you also need to distribute OpenVPN binary with each Mysterium Node, which involves two steps and is never great for UX. It was workable for a proof-of-concept or very early versions, but as we moved to mobile platforms, this approach became very complicated or unfeasible, especially when considering iOS.
To solve this challenge, we decided to find a way to integrate OpenVPN into our Golang project directly. Also, we decided that this package could be useful for others — that’s how the Openvpn3 library was born. Openvpn3 is the official library maintained by the OpenVPN team and is being used on almost all platforms as client or connector to the OpenVPN server. Also, it’s written in C++ which came with some obstacles we needed to solve.
Our first obstacle was that C++ code cannot be directly called by Golang (Cgo to be exact). We needed to make small changes to the OpenVPN library itself to export OpenVPN Client as C callable code. Then there is how Golang treats C code itself. The problem was that Golang and its package management systems expect that all libraries are source files (i.e. there is no, or very limited, binary package management). And the Openvpn3 library build process was very complicated and not easily expressed in a Go way.
SEE ALSO: The 2019 State of API Report says standardization is the #1 challenge
So our decision was to compile that library in advance for all platforms we currently support or produce binaries for arm family (Android, iOS), AMD64 family (Windows, Linux), some simulators). As we use Linux for our automatic build system, we had to set up all compilers and SDKs in one place. The result was a single header file and a bunch of static libraries for each platform/OS we needed. We also had to ensure that these binaries were Go gettable (the go way to fetch a library from Github).
We simply committed those libraries to Go repo along with all supporting Go code. This was not the best way to distribute the software, but our target was a Go gettable library. Next came the easy part: to call Openvpn3 functions from Go. It was quite easily doable. However, there were a few problems. First of all, strict rules as to what can and cannot be passed to C code and vice versa (for example, you cannot pass Go function reference to C code). The Openvpn3 client also depends heavily on callback functions. One way to approach this was to use only static functions for callbacks. However, this would have limited the flexibility and usability of the library. A hybrid solution was to define customisable callback functions in Go and register them in a map with function ids. Static functions in the Openvpn3 client would then dispatch respective callbacks to registered functions with corresponding ids.
Here is how it works (let’s take state event callback function as example). First, the user defines normal Go structures with methods, which satisfies interfaces expected by callback registry. Then, structure is passed to callback registry which is essentially global id to callback map. What happens next, the callbacks registry inserts user provided structure with methods, and creates a C structure, ready to be passed to C code, but instead of passing Go function reference to C code, it passes id which is simply key to callback map and an exported go function (with special comment).
When C code wants to inform user of state changes, it calls static Go function and one of the parameters is id. That id is then passed to callback registry to find and call appropriate user defined callback. After this, it is compiled. At least the Go part – that means that C code is reachable, and all headers are ok.
SEE ALSO: Golang is the highest-paying skill in the US, report shows
Most of the dragons started rearing their heads when it came to linking the Go packages with OpenVPN static libraries.The biggest issue was that: the library was built with C++ compiler, but Golang cgo used C compiler by default. As a result, all weird and ugly errors began to arise at the linking stage. So, if you see similar errors as in example – you are not alone. After hours of stack overflow exploration, a simple workaround was to put a empty. That way cgo was tricked into using the C++ linker which already had C++ library by default.
When using new technologies like Golang you have to sometimes go off-chain to find solutions that will help you use existing libraries so that you don’t have to start everything from scratch. However, as with most solutions in IT, it’s not a silver bullet. For example, precompiled libraries on their own poses a security risk – potential library users cannot be sure what is exactly compiled therein, meaning there is no code to review. Each OS and architecture combination has to have separate versions of the same library. Also, there is the iOS framework problem – iOS framework lib (provided by the Gomobile tool) is a static library itself. So any other dependencies are linked but not combined into a framework – and this needs to be done as a separate step. It’s simply not a Go way – Golang usually expects all sources needed for a package to be in one place.
The post Golang and C++ interoperability: The reincarnation of OpenVPN’s C++ library appeared first on JAXenter.
Source : JAXenter
