New C++ Standard, New Focus
October 31, 2011 16 Comments
Now that the new version of C++, C++11 is out in the wild as an ISO C++ Standard, there’s a renewed focus on getting applications, tools, and books up to date with the new standard. The leaders of the ISO Committee have weighed in on the latest versions of the language. There’s Herb Sutter writing about C++11, there’s Bjarne Stroustrup’s C++11 FAQ, and the upcoming C++Now! conference that’s going to be held in Aspen Colorado. Clang is racing to get C++11 support into the stable release of the compiler, and GCC is humming along nicely with it as is Microsoft’s Visual C++. Instead of write about any specific things, I would lend my opinion on what needs to be the new focus not for C++11 the language, but the supporting toolchains surrounding this new language — and the version that’s coming after this one.
Having been exposed to a number of programming languages and paradigms I’d like to think that I’ve seen quite a bit of the programming world. There’s the dynamic programming languages like Python, Ruby, Lua, JavaScript, and its ilk, and there’s the relatively new static languages like Go, Scala, Haskell, Erlang, among others. Systems programming and application programming have new challenges — concurrency, consistency, safety, etc. — that need attention from both the programming language and the libraries that come with the programming environment. There are new platforms that are designed to perform more and more demanding computations in very meager amounts of available resources. The web is ever-growing and there’s a lot more things web applications need to do both at the front and back-end of the system. And there are yet a lot more applications to be thought in embedded and large-scale systems that current programming languages have yet to even think about (autonomous computing anyone?).
With these in mind, I think C++ has to evolve into a direction that makes simple things easy and hard things possible. There was a time that C++ was the language of choice because much of the infrastructure that’s available now was not available then — you kinda had to write your own tools back in the day, and C++ was the easier programming language to use for large systems. Unfortunately now, even with the new features that makes it easier to write programs in C++ using the new modern style, a big percentage of the programs still turn out to be either glue code or repetitive boilerplate. If you’re writing asynchronous systems, they’re made easy with lambda’s, efficient containers, consistent memory model, and concurrency primitives (or at least standard libraries) but you end up with a lot of boilerplate because there aren’t any sane standard HTTP implementations — interfacing with C libraries will be a lot more painful with C++11 considering how much of the C++ standard defines things outside of the C standard’s scope. C++11 is awesome if you’re starting from scratch but when you get to the part where you have to tie libraries together, C++ is still an expert’s language.
I’m hinting at two things here that C++ should address in my opinion going forward: a standardized way of defining modules for easier composition and higher level standard libraries with reference implementations. Let’s tackle each in order here.
First, standardized modules. The reason programming languages like Go, Haskell, and Erlang are getting traction for back-end programming (in the case of Erlang, it’s not really a “systems programming” language unlike Go and Haskell) is because it has a simple and standardized module mechanism. Packaging code in modules that can be programmed against (ala Pascal Units) in terms of interfaces makes modular development really easy. Right now all the module implementations in C++ are vestiges to how the linkers are implemented and implementation specific. There’s a lot of details here that’s worth fleshing out at a later time (there’s a lot of content for the blog) but needless to say it should be simple to import modules into a compilation unit for later linkage — and this should be made standard across implementations.
One of the problems that standardized modules will solve is that of the definition and implementation of the standard library. The STL is awesome as it is at the moment, but it can be a lot more awesome if there’s a defined module layout that will be the same across all platforms and that people can implement shrink-wrapped modules that play well with many other modules in a non-conflicting manner. C++ already has namespaces and it’s a mechanical manner of tying namespaces to modules. Extending a module should then be a matter of dependency and linkage. Without going much into details, C++ needs to solve the problem of coupling. Many things need to be thought out, like how the runtime environment deals with types and namespaces and how linkage and application binary interfaces can be made consistent without sacrificing performance.
The second thing that does need attention is that runtime environment. I’m not just talking about garbage collection and I’m not talking about having a virtual machine. What I’m talking about is the management of runtime elements, things like memory, I/O, concurrency, among other things. Although it is not impossible to write OS kernels in C++, the C++ runtime is mostly still the C runtime (i.e. too limited) to make things like a global optimizing allocation mechanism, thread scheduling, implementing co-routines, and magical things that makes systems programming interesting very painful to do. A standardized runtime environment can make writing stand-alone systems in C++ easier and less reliant on the C runtime. This can also finally solve things like module initialization (when dynamically loading things), static initialization, memory management, and even reflection and runtime introspection features that can be implemented in a standard manner.
The runtime should be as programmable as the Lisp environments, as minimalistic as the Lua runtime, and as performance-sensitive as the C runtime. It should also be as open-ended as possible, still allowing for implementation-specific extensions but defining a standard core that others can build upon. There are a lot of things to flesh out in this idea but it’s something that needs to happen to make C++ a language that’s as viable for systems programming as well as application development with all the goodness of the dynamic languages and the safety of static languages. Short of defining a virtual machine, a standard runtime implementation should definitely be considered for the next version of C++. I don’t like virtual machines as much as the next person but having a well-defined runtime environment will make systems programming a lot less hack-laden as it currently is.
With these in mind I’m putting some effort into writing up papers to that effect so that hopefully these ideas get at least thought about in the next ISO C++ Standards Committee meeting in Kona Hawaii. Hopefully I can get comments and ideas and a discussion going with esteemed colleagues and users from around the world who might have an interest in joining an effort to make this a reality. Let’s make C++ even better by bringing good ideas from other programming languages into C++ without sacrificing the performance and flexibility afforded by our favorite programming language.
Along with this effort I’m going to work on getting the C++ Network Library reviewed for inclusion into Boost and hopefully be a reference implementation for a high-level network library to be made part of the C++ standard. I think it’s about time that C++ gets modern programming features and Web-aware libraries part of the standard to make it more attractive to developers who might want to develop high performance solutions in C++ for the Internet. I’ll be supporting the standardization of the Asynchronous I/O library that’s already part of Boost so that it can be the standard way of programming asynchronous I/O in C++ and made available to more people who do use C++ for network programming.
It’s an exciting time ahead for C++ and I fully intend to help influence that future too.
About Dean Michael Berris
>…and the upcoming C++Now! conference that’s going to be held in Aspen Colorado.
Will you take part (cpp-netlib)?
>…ala Pascal Units…
ahh… plain old Turbo Pascal (and Turbo Vision). Even it sometimes needs *.int files to hide implementation from interface.
> A standardized runtime environment can make writing stand-alone systems in C++ easier and less reliant on the C runtime
It’s hard to not starting conversion of C++ into one more Java-like language (instead of using advantages of each platform Java supplies platform itself). Won’t support of rich runtime kill C-nature of C++ (where we aren’t required to pay for the features we don’t use)?
IMHO:
C++ is the king among non-managed programming languages (Go?), if someone needs managed-language specific (wide runtime support, GC) then he would switch to existing managed language, i.e. the choice lays not between C++/Java/C#/JS/etc but between non-managed/managed language and static type system vs dynamic one.
> I’ll be supporting the standardization of the Asynchronous I/O library…
Could you explain more?
I’m not entirely sure yet whether I’m going in May, but I may submit a paper that I may ask someone else to present (regarding cpp-netlib in C++Now!).
Also, when I say “standardized runtime environment” I’m talking about features in the language that specifically address the presence of a runtime. Things like introspection of types/pointers at runtime, the potential replacement to ‘malloc’ that acknowledges that this program is written in C++ and that there may be type-specific information that’s important to determine behavior of the allocator at runtime, and system-level things like primitives for accessing machine state. I’m not talking about a rich runtime. Maybe I’ll write more about this idea in the following weeks/months.
So to answer the question, no, having a well-defined runtime environment that is consistent across platforms does not mean sacrificing the minimal C-like runtime environment. Making some things consistent across platforms can be done to even potentially make certain things a lot more efficient (consider a standardized runtime that defines how subroutines, co-routines, tasks, processes, and threads are defined at a lower level, etc.). No “managed runtime” required.
As for ASIO in the STL, I have heard of efforts to get the ASIO interface included into the STL. I haven’t found the papers yet, but the LWG is meeting soon and I do hope to see a paper to introduce it into the standard in TR2 — if there isn’t one, I’ll be willing to help write it with those who would be interested in supporting it as well.
Thanks for the comment!
“Clang is racing to get C++11 support into the stable release of the compiler, and GCC is humming along nicely with it as is Microsoft’s Visual C++.”
Please get your facts right.
GCC is still the compiler with the most C++11 support. Clang is barely usable for any C++, 03 or 11 (it ICEs all the time).
MSVC11, on the other hand, doesn’t really add anything new compared to MSVC10.
MSVC is still the barrier that is today preventing large-scale adoption of C++11.
I recommend giving Clang another spin when version 3.0 is released in a few weeks. We’ve found that it is quite stable and enjoyable to use. Bug reports to the contrary are quite welcome.
Regarding C++11 support, one can easily compare GCC’s support
http://gcc.gnu.org/projects/cxx0x.html
to Clang’s
http://clang.llvm.org/cxx_status.html
Objectively speaking, GCC supports more C++11 features than Clang. Clang is moving fairly quickly, as noted by the author: just compare the “version 2.9″ features to the “version 3.0″ features (there are ~6 months of development between 2.9 and 3.0).
There is information about the new features MSVC11 will have, here:
http://blogs.msdn.com/b/vcblog/archive/2011/09/12/10209291.aspx
Which version of clang do your facts concern?
Latest SVN.
Then what is your definition of “Any C++”? I have used clang on various C++ code and even build clang itself without seeing any internal compiler errors.
By “any C++”, I meant 03 or 11.
I didn’t mean “any C++ code”, but rather “any C++ version”.
Of course, if your code doesn’t make heavy use of templates or low-level features like vector intrinsics or special attributes, you should be fine.
Great blog. Could wish for bigger font though.
One thing I’d like to see revamped is boost. Right now the way boost works is very outdated.
It’s one huge monolithic library. Getting new projects into boost takes a long time and requires a lot of review. For that reason, boost is missing a lot of useful functionality. Many of the best c and c++ libraries are not in boost.
Also, as a developer, I often don’t want to use boost because it is difficult to build. Usually if I try to build the whole thing, at least one library will fail. Many people stick to the “header only” libraries in boost. However, some of the more useful libraries are not header only, such as the regex libraries.
Anyway, what the C++ community really needs is a package manager. Any modern language like python, node js, ocaml, perl, etc, has a package manager that makes it easy to distribute libraries to new users.
Package managers automatically pull in transitive dependencies, build, install and update libraries.
Since package managers aren’t distributed as a single monolithic download, but let the users pick and choose what to install, the barrier to putting code in a package manager can be much lower.
I think this has been thought about before, and there’s an effort to do this called Ryppl which aims to be applicable to not only Boost but for other C++ projects as well. This would be a lot easier though if there was an easier way of using proper modules in C++ from a language perspective. The reason this works reasonably well in Python and Ruby for instance is that because they can isolate modules properly as independent pieces that can be distributed with explicit dependencies to other modules.
Thanks for the comment though, and I agree that C++ needs a CPAN/RubyForge equivalent to get more open source libraries easier to use and more accessible to more people. Haskell has the same thing with Cabal and Hackage, and I don’t think it’s impossible to achieve with C++. Someone has to do the work though and it is a lot of work to maintain this infrastructure and get the myriad C++ libraries already out there in a form that is easily distributable across platforms.
I agree that C++ would be much better off with a real module system.
I think a package manager would need to be smart enough to deal with legacy code though. After all, there’s a lot of libraries built on top of the existing system that people will still want to use.
If you look at debian, red hat, arch, etc, they have package managers that work with plain old c code. Dependencies have to be explicitly specified in some kind of package manifest.
I agree though. The real limitation is that someone has to actually do the work.
I’m not sure a package manager for C++ is unnecessary. Most programming shots in the corporate world don’t like using other people’s code, although we’ve adopted Qt. Then again, on the flip side, Google itself works pretty good at finding source code.
On a side note, CPAN isn’t all that great. A lot of the code in there is really old and broken, which is why we ditched perl for C++, it seemed a lot of things relied on borken CPAN modules.
I guess my point is, I like C and C++ because it sort of forces me to code my own stuff, then at least I know it works.
A lot of the ex-perl users I know have switched to python. I think that has a lot to do with why perl packages are getting outdated.
My feeling is that coding infrastructure from scratch is good for educational purposes, but bad for business.
All software has bugs. If you write a lot of custom infrastructure that gets used in production, say an HTTP client library, developers are going to have to continue fixing bugs in it for the lifetime of the company. Eventually the guy who wrote it will leave the company and the new people maintaining it won’t really know the code base that well, so it will start to get messy. Also, no one will bother to write documentation for an internal tool.
On the other hand, if you use a good open source library the maintainer is much more likely to stick around over a long period of time. Even if he changes jobs, he’s likely to continue working on his pet project. The code base is also likely to be tested by many more people than is possible in a small company, so you won’t have to fix every bug. Finally, since an open source library has external consumers, the developer is much more likely to provide some level of documentation.
The main thing to keep in mind is that a software business’s job is to produce software, not code. Software is something you sell to a consumer for a profit. Code is something that costs money to write and maintain.
Lots of good points here. Amen to the discussion on standardizing modules.
Small point of clarification: Erlang is a dynamic language, not all functional languages have to be static
Dean
I’m interesting in this subject quite a lot. I’d like to ask your opinions privately about something I feel is related to this subject. It’s just to help me clarify some thoughts but you might find it interesting too. If you find you have the time, can you drop me an email. No worries if you’re busy, I’ll completely understand.
Thanks