Costs/Funding in Open-Source Languages

Show and Tell
1

At some point I hope to do a talk or blog post about “the economics of programming languages” to talk more about all this in depth, but here’s an overview of some of the things I’ve learned since “Let’s be Mainstream” when I knew a lot less about how organizational structure influences who ends up using a language.

Developer Relations

Here’s a job listing for a community engineer for Rust. That’s probably a $100k+ role. A full-time job that requires both technical expertise and great communication skills. Many people will apply, but they will choose a certain person who they think is best suited. If that person gets burnt out, they quit and Mozilla pays someone else $100k+ to do it. This is roughly the job that is most commonly suggested for Elm.

Wait, how does that open-source projects have the money for that?!

Common Funding Methods

Browsers are mostly funded off Traffic Acquisition Costs (TAC) where Google uses some of their ad revenue to pay to be default search in browsers. In 2019, they paid about $30 billion to companies like Apple and Mozilla Corporation to be default search on iPhone, Safari, Firefox, etc. That is about 20% of Google’s revenue, but it was as high as 39% before the Chrome project. Chrome existing and having high market share saves them billions of dollars a year in TAC, and they invest some of that in implementing new features in Chrome. Outside of browsers, voting seats on TC-39 cost $70k a year and are often occupied by people paid to work on browsers. The $100k+ DevRel salaries come from these massive flows of money, and that facilitates the interactions many people expect from Mozilla and Google.

I do not think operating-system funding works the same as language funding, partly because it is more culturally acceptable to do a consultancy like at Red Hat or Ubuntu in that field, but there is still valuable information there. E.g. Linux usus a membership model, partly funded by “platinum members” like Tencent that get board seats in exchange for their financial support. Or IBM that says here:

“We’ve been one of the bedrock members of The Linux Foundation since its inception. We have a long history of doing open source projects throughout many communities. (But) there are things we couldn’t do before that we can now do, working in partnership with The Linux Foundation. “If we were to take a project, open source it ourselves, and expect people to come to that project, that’s a very difficult path. But when you do it in partnership with The Linux Foundation, that path gets smoothed.”

Running an organization the same way as these massively-used projects means selling things. Maybe it is influence. Maybe it is rebranding. Maybe it is default search. Etc. If you are prioritizing adoption very highly, it makes total sense to do these things! How can we settle disputes between Apple and Google about something in Linux? When some new company starts competing with them, what about them? Membership models answer these questions really well, and it gives you budget to have dedicated developer relations roles. It comes with trade-offs though.

Desgin Incentives

When you run a project in these ways, you typically have more incentive to say yes to features. Say that a segment of users wants a |> operator in JavaScript, what would be best for Google? Option 1: reject the proposal. In this case people will be mad at Google. People may even create elaborate theories as to why Google was ill-intentioned in this decision. Maybe they start talking about the Traffic Acquisition Costs are kind of weird, especially considering that Bing could put its entirely yearly search advertising revenue of $7.6B towards TAC and still not afford being default search in Apple products for a even a quarter of the year. Option 2: accept the proposal. The language will get more complicated and confusing over time, but that conflict manifests within teams. Now coworkers are debating about style guides and linting. They are mad at each other, not at Google. If I was a risk averse VP of Engineering, I’d be leaning towards accepting as many proposals as possible.

And if you want everyone to use your search engine, having a design process that biases towards adding features makes sense. Everyone can have what they want as a subset and keep happily using Chrome! But I do not think all technical artifacts can be constructed in that way. Over time, you end up with more and more features, but the whole point of some projects is to have fewer features to create a more coherent experience than in projects from the big companies.

Alternative Paths

I know volunteers in open-source can have their project get to a certain point where they start facing questions about how to prioritize. If you only have nights and weekends, do you add that feature or spend time with your family? Do you try to be a volunteer Community Engineer, or do you go to the gym to work on your weight? I have thought a lot about how to support folks like that in a full-time role so they do not need to make trades like that to meet the expectations of some users, but even if there was a magic wand to get money for that, I’d want to be extremely careful about the long-term incentives it would create.

People are trying funding independent languages in new ways, like Kotlin+IDE and Clojure+consulting and Unison+hosting, with the hosting folks typically having some VC investment. These all have trade-offs as well. Even if it was possible to snap to instantiate any of these models into reality, you are going to alienate people who implicitly prefer TAC funding where you never really need to think about who pays or why, or those who prefer membership models where they know they can pay to prioritize work if it ever comes to that. But what happens when IBM and Tencent pay to prioritize work over many years? There are very serious trade-offs on every path, and paths tend to be quite sticky.

I know Python has shown that you can run community events through a Foundation without creating strong language design incentives, and I have been working since February on getting Elm Software Foundation in order to do something like that (travel grants, Elm Bridges, conferences, etc.) I found a great lawyer to double check things, worked on figuring out accounting, coded up donations via Stripe, etc. I think this is a promising possibility, but since it primarily focuses on supporting community events, I felt that it would be best to present this work after the global quarantines and travel restrictions start to ease up.

Conclusion

I am not saying any other project is running things wrong or that we are running things right. Every model comes with costs and benefits. Every model has different incentives and produces different technical artifacts and communities.

My point is just that people seem generally unaware of how much certain work costs, how it is typically funded, and the incentives that come with the common funding methods.

I hope that subdividing projects by category (language, operating system, package, editor, etc.) and by funding model (TAC, app sales, membership, benefactor, consulting, conferences, etc.) will help people make more appropriate comparisons. Whatever path you take, it will come with obvious benefits and non-obvious downsides. I do not have all the answers here obviously, and I am happy to talk with people who have serious experience with other models to learn more about what can be adapted, what it might cost to do so, and how the relevant relationships were developed over time.

48 Likes
2

I was scheduled to talk about some of this at Deconstruct Conf in April, but all that got cancelled.

I was also scheduled to talk about “Elm for Science” and “Elm for Beginners” to share some of the experiments that I worked on late last year and ask for help. I think those still work best as a talk, but it may be worthwhile to convert them into blog posts instead.

Anyway, I was motivated to share this by a recent conversation. Perhaps it’s valuable to share that as well:

evancz
One thing I may end up talking about more in public is that many projects have $100k+ roles to handle the volume of community interaction that comes to the project. For example, there was a recent job listing with Rust for these exact kinds of tasks: Rust Community Engineer — Mozilla Jobs — Open Positions

There are many reasons that Mozilla Corporation (Mozilla Corporation - Wikipedia) can afford that, and that other organizations cannot afford that kind of thing. I think most casual observers and users of open-source are pretty naive about what is for sale with some projects. (E.g. voting on the EcmaScript committee costs 70k a year Join Ecma - Ecma International) It definitely took me many years working as an author to understand this clearly. Anyway, I hope that talking more about things like Google’s “Traffic Acquisition Costs” will make it clearer why the expectations people have for projects from Facebook, Microsoft, Google, Apple, or Mozilla may not make as much sense for more independent projects. (Google spent about $30 billion in 2019 to be default search in Apple and Mozilla products. That money in turn goes into the $100k jobs I am talking about. It also makes it too expensive for Bing or DuckDuckGo to be default search. Bing’s yearly revenue of ~$7B is not enough to be default search on Apple products for even 3 months!) Being connected to the biggest ad networks on the planet comes with many obvious positives and many less obvious negatives. (edited)

ABC
Have there been businesses besides NoRedInk express interest in funding open source Elm development?

DEF
@evancz would it be feasible or sensible to have the community contribute in some way? For example, I know there’s the Elm Software Foundation. Does it accept donations? Could a crowd-funding project be set up to help with it?

On the other hand, could sponsorships be asked/donated by the companies using Elm in their projects?

I guess all of this would have to come with some kind of transparency so that contributors (be it people or organizations) know where their money is going into

evancz
There are lots of models of how to set these things up.

I did some work in February to allow ESF to accept donations online. I also coordinated with some people to see what they thought of crowd-funded work. Everyone I talked to had very serious reservations about having their livelihood dependent on a per-project-crowd-funding model, so I was planning to start with a focus on community organizing, like Elm Bridges, travel grants, etc. Coronavirus led me to delay moving forward with that project since it would focus on in-person stuff. Maybe it would grow into something that could support engineering work, but we’ll see later.

Community donations for engineering work is basically not used by programming languages. Maybe it can be done, but Python, Clojure, Haskell, JavaScript, TypeScript, Ruby, etc. etc. do other things. A lot of compiler work can be quite risky and long-term, which I think makes it a sort of poor fit for crowd-funding, so it’s interesting to see empirically that most projects do in fact do other things.

I wrote a bit about these things in 2019 here: news/elm-and-bekk

evancz
One thing to note is that doing administrative work to evaluate and work towards something here is in direct conflict with working on other things. This is part of why projects big budgets coming from Traffic Acquisition Costs hire people to do community-facing work. I don’t know the churn on those kinds of roles, but my instinct is that it’s higher than the compiler roles due to the nature of emotional labor.

ABC
Are there schools or programming language research groups that could be willing to partner? Grad students could maybe do experimental compiler work as part of their thesis

evancz
All funding means “selling” something. With the academic funding, the language really needs to be a good “research vehicle” so that people can publish papers and show impact in practice. For example, bringing “modular implicits” to OCaml or dependent types to Haskell. If that gets popular in Haskell or OCaml, maybe it finds its way into a Microsoft language some day. (E.g. Philip Wadler’s 1988 work on type classes probably had some influence on his work on Generics in Java 5.0 in 2004.)

Anyway, these languages tend to have lots of advanced features specifically because they are excellent research vehicles. So I think that kind of partnership can only really work if you are down with the trade-offs of being a research vehicle.

evancz
I know compiler teams will hire PhD students for internships, and the desirability of that for students is usually dependent on their ability to get a paper from it. My impression is that Microsoft has a pretty good reputation for getting research papers out of PhD internships. I’m not sure about Apple. I’ve met people doing a masters in PL that work on Swift or Clang, but the goal with a masters is more to do implementation work. (You can find lots of people with PL PhDs on JavaScript VMs. That can be a cool path if your PL work is more oriented towards object-oriented and/or dynamic languages.)

ABC
How about computing education researchers? If a goal of elm is to become a best language to learn programming with (it probably already is lol), then there would be a big alignment of incentives.

evancz
That could work, but there are risks. For example, there is a professor at UChicago that has done a data structure course with Elm, and he has worked on GitHub - ravichugh/sketch-n-sketch: Direct Manipulation Programming for HTML/SVG which could potentially be a way to do “direct manipulation” in Elm. I.e. you right-click some text and can mess with the color, and and that’ll update in your source code!

But the incentives of academic work are to get the paper, so there’s often a great deal of “to the finish line” work that does not end up happening. Doing the “to the finish line” on direct manipulation in Elm would be a major task, both creatively and technically. How general can it be? Can it be less general, but in a way where I can still get NSF or DARPA grants? Etc.

ABC
True, but at least it wouldn’t cost any money

evancz
I know there are PL people who are interested in empirical research into usability, so there definitely could be something there. But I don’t actually know who is doing that or if they’d be interested in doing it with Elm. Investing in building those relationships is time not spent answering issues and pull requests, or working on ESF, or working on the compiler, etc. etc. So I have to think if the risk is worth the reward, especially if their core incentives are oriented towards publication.

One cool thing about that sort of path is that anyone who knows people in that sub-field can talk to them about the possibilities here. It doesn’t have to be me. One of the major differences between “corporate open-source” and “independent open-source” is that the big guys have money to pay someone full-time to think of these possibilities and work on developing the appropriate relationships. With more independent projects, those sorts of relationships tend to be formed by individual initiative of community members.

Slackbot
Terms like guys can make people feel like they’re being left out! :scream: Let everyone and all the folks know you’re talking to them! :heart::robot:

evancz
Haha, “the big multi-national corporations” :smile:

ABC
Do you have any specific projects in mind that those kinds of researchers can help work on? (edited)

evancz
With people working independently with their own incentives, a good project is usually only discovered by them or by talking with them.

With any volunteer, they know their interests, constraints, skills, goals, etc. better than you, and those are all really important in creating a viable project for them. But with academics specifically, they mostly get into research because they want to look into what they find interesting. There’s lots of factors that go into that, and you need to be familiar with a given sub-area of PL to create interesting proposals. It’s the whole task of a PhD student to think of something interesting that no one else has thought of!

XYZ
Evan, I think what you said in this thread could be a very useful blog post, not only because it would be informative about open source funding and contributions, but also because it could highlight a reason to choose Elm.

For example, I have reservations about using anything from ad-revenue corporations like Google (eg Flutter or Dart) and Facebook (eg React or ReasonML), and I’d appreciate if a project specifically highlighted its independence from them. I think quite a few developers share these reservations.

I anonymized the participants a bit. I am speaking casually in these chats, so there may be more mistakes than in other things I share publicly. If I were to try to express the core ideas here more publicly, like in this Discourse or especially in a blog post that goes to folks who do not use Elm, I would want to work extremely hard to say things better. Am I saying something that could be taken the wrong way? As an insult? As an excuse? As a misunderstanding? Etc. Over time, the risks of talking in public have gone up and up. It’s much easier to get thrashed! So this thread is sort of a trial to see how it might go to start bringing these things up.

I’m not sure I’ll end up talking much about academic funding and incentives otherwise, so I figured sharing the very casual chats may be the only way. I definitely may be making mistakes my characterization, as I have never been a PhD student or done any NSF or DARPA grants myself!

34 Likes
3

A note on using Rust as an example: while it’s technically correct that they are funded by Mozilla and whatever income they rely on, Rust is also a strong example of those costs having little effect on the direction of the language design. Aaron Turon wrote an excellent series of introspection posts on a few times where members of the Rust teams went around the community, thereby upsetting a lot of people. Today, Rust is extremely community oriented, which helps keep the economic interests of individuals from having too much impact. So I think it’s important that you can make money without compromising on design because of that income.

There is also some work to start a Rust foundation; this post from Niko Matsakis outlines a lot of interesting points in that regard, and I would also recommend his post on where Rust should go in 2020, since it talks a lot about how to help others invest in Rust.

8 Likes
4

At its peak level of support (roughly 2016-2017), Mozilla funded I think 9 FTEs on Rust, multiple summer internships, and a substantial contracting budget. That level of support has been in steady decline: its been 3 FTEs since February 2019 (I am no longer one of them), this hire will bring it back up to 4. And the contracting budget has also declined a lot, though I don’t know the exact numbers. I’m not sure how many FTEs are expended on Elm, but maybe the funding gap is not as great as you imagine anymore.

But Mozilla has really, truly never intervened in Rust’s development, except I think possibly to put some pressure to release a 1.0 in 2015 (I wasn’t around then, but that’s the sense I’ve gotten). While I was at Mozilla, I had two skiplevels at different times; I can’t remember either of their names, because I never interacted with either of them. The Rust project’s priorities were determined entirely internally to the Rust project. I genuinely don’t believe there’s anything the Rust project could do now that would change how much longer Mozilla will fund it and at what level, for better or worse.

Differences in direction between Rust and Elm I think largely derive from psychological factors. To be blunt, I have said before that Rust has a culture of “accommodating assholes,” and I get the impression Elm is much better at not doing that. I think that is wholely the result of different attitudes held by the individuals in leadership in each project over its history. Different people have different levels of comfort with disappointing people, different sense of obligation to their most vocal users, different amount of self-confidence in their decisions, etc.

I also don’t get the sense that Google’s interests drive the design of Go in precisely this way either, to look at another example of a language that has full access to the adtech moneys. In the case of both Rust and Go, I think their sponsors fund them for essentially the same two reasons: 1) these languages meet a need the company has, 2) associating their brand with the brand of these languages and their creators is good for their developer relations. I even think 2) is the larger reason most of the time, which is why no company wants to be percieved as “meddling” in the language’s development. That’s disastrous for the brand!

At the same time, when I look at a language like JavaScript or C++, that seem to grow indefinitely as they shift from paradigm to paradigm, I see a specific cause of that: they are a shared platform for many stakeholders, which people are unable to migrate away from and unable to break, but which people want to have modern features they believe will make their experience better. When you can only choose two out of new features, backwards compatibility, or constrained size, its obvious constrained size is will lose out. I personally would prefer to see a world in which we shift to new languages as we learn new techniques for defining them, rather than carrying the weight of history on our backs, but backward compatibility has a strong pull and I am not surprised there is money in continuing to extend existing platforms.

So I don’t think financial incentives have the explanatory power you’ve suggested here, except perhaps that companies will more readily expend money to keep extending the tech they use than to build new tech to replace it. I think there’s just too many steps removed between programming languages and how the big tech companies earn their money. I personally would be eager to siphon off a bit of that ad network revenue for the tech I care about, the problem is convincing any of the people with their hand on the spigot to continue to invest in you over a sustained period of time (as Rust is now discovering).

On the other hand, I couldn’t agree more strongly that many people seem to be orders of magnitude off on how much developing good technology costs, and very ignorant of the different forms of labor involved! It doesn’t help that open source tech is a cost center that’s given away for free to all competitors.

35 Likes
5

I am excited to see your perspective on this! I have a feeling we might agree on pretty much everything, but I need to share a bunch more to clarify what I am trying to say.

My belief is not that the interests of the big company directly determine language design choices. I have never heard any stories of this kind of thing when talking to anyone paid to work on a language, and I feel like I would have by now. My point is that I think people can set better expectations for a language if they think about the costs and funding, and how having headcount at all already filters for specific characteristics depending on the organization.

I am feeling reckless, so I am sharing a bunch of ideas that I have not talked about publicly for many years. Sorry this is so long.

“Big Company Logic”

I heard a headcount story where someone very justifiably wanted fewer engineers on their large team, but they did not think that would happen. This person led me to believe that in their large company, having high headcount meant you were important, and that important people do not have low headcount. I was very confused by this. If you think you can a better product with fewer workers (i.e. less money) why not do that? The best they could explain was “big company logic :man_shrugging:” I have thought about that story a lot since I heard it in 2016.

This story has informed my perspective on Google, as they seem te be a pretty unique case when it comes to languages. Here’s my understanding:

  • Go - Before there was Go, that team worked on a language called Sawzall that was all about processing logs. This may not sound so important, but the way Google calculates how much people owe them for ads is by processing data like this! So my understanding was that this was something of a “count the money” project that was extremely valuable. My sense was that this team was highly regarded in the organization.

  • Dart - Before there was Dart, there was a SmallTalk team that made a really fast JIT compilation VM, and later incorporated those techniques into the JVM. Some fraction of that crew were brought into Google to implement fast JIT for JS in Chrome. The great perf was a major part of how Chrome gained market share. Going back to the TAC theory, this whole effort saves Google billions of dollars per year, and that team is very highly regarded in the organization.

I do not know the headcount numbers today of these projects, but I know Dart once had a much higher headcount than the highest numbers I have heard for Go or Rust or Swift or Elm or Python. So I personally came to believe that these details are important to some degree at Google. I am sure there are a whole range of factors, but my sense is that Google has a bit of a “you did good, make what you want” vibe that is maybe more lenient than at other big firms. (Thinking of GoogleX as well.)

Putting that aside, I did not get the sense that “developer experience” was super accounted for in the org chart, and that work focused on developer experience happened in a fairly ad-hoc way. The project manager who turned the “IE Toolbar for Google Search” into Chrome eventually went on to become CEO, and I personally think he has a very keen eye for how free consumer products scaffold their position regarding ad revenue, much more so than how developer products matter for their business.

This more ad-hoc system is in major contrast to Microsoft which has specific departments that are meant to work on languages. You have Microsoft Research which has supported tons of very interesting PL work, and that work often ends up manifesting the great work from their Developer Tools department. I suspect they have such a well-organized and well-funded organization from the Devolpers! Developers! Developers! era, but I am not sure how all this fits into modern Microsoft with Azure and all that.

I think adding “big company logic” helps explain why various languages have headcount at all. Again, I have never heard a story of people outside of the project influencing specific design decisions, but I think it would be a mistake to think that means things like TAC do not matter. I learned about “which projects have headcount at all?” when smart phones were getting popular.

The Spread of Smart Phones

I personally noticed that a ton of open source initiatives began when smart phones started getting really popular. Mobile games were getting huge, and users of Facebook/Google were moving to mobile apps but the big companies didn’t have their mobile ads networks figured out. It became “mobile first” because everyone thought they were facing an existential threat. I remember people saying things like “Who needs browsers when you have every app you need?” and “Why settle for JS performance when you can get ObjectiveC performance?” A lot of people were freaking out at this time. For example:

  • How will Prezi survive in a mobile world without Flash? Do we now need teams for Android, iOS, browsers, tablets, and desktop? Can we share code? How do we keep things in sync in the product? What if one language could compile to many platforms? Is it worthwhile to hire one weird kid as well as all these other people and see what happens? So I don’t think they would have offered the no-strings-attached support that they did if they were not facing so many risks at that particular moment. They also did a short contract with another compiler project at the time to see what’d happen there. Elm was too young for what they needed and the standard just became to have a bunch of teams, but I do not think Elm would exist today without this support.

  • How will Mozilla survive the transition to a mobile world? If over 90% of revenue is TAC, what happens when traffic is all on Android and iOS? What about in developing countries where it is only mobile?! This all coincides with projects like FirefoxOS and asm.js, which could conceivably preserve some sort of TAC funding if the worst case scenario came to pass. “Maybe we cannot be on mobile in the developed markets, but maybe we can focus on developing markets that have lower powered devices?” People had been talking about having bytecode for the JS VM for many years, so I found it interesting that projects like this started at this particular moment.

The absolute worst case scenario did not happen for Firefox, but it’s position was weakened by this whole era. Google came out much more powerful, and Chrome continued to gain market share in browsers and on phones. My total outsider perspective is that this seems important for Rust headcount. Having safe/secure concurrency could mean Firefox could deliver features faster and use less memory than Chrome. If it could do that, maybe they could start pushing market share back up and negotiate for better TAC.

So far this does not seem to be bourne out by the market share numbers, so I think this overall theory fits the details you shared of how headcount changed over time. A friend of mine worked on FirefoxOS and I think headcount there followed a similar path. I am not saying this to criticize Mozilla or Rust at all. When there are 1000 people in an organization, it would be totally reckless to have no strategy for how you figure out headcount for all the various open source projects within the organization. Mozilla definitely cannot fund them if it does not exist any more.

When I started looking into earlier times to try to understand what was happening here, I started to get a feeling for some of the factors that influence “open source culture” as well.

“The Browser Wars” and Open Source Culture

Going back to the 90s, Microsoft was the dominant force online, and this created enough bad feelings that it turned into a successful anti-trust case in the US. The major advocate for the term “open source” was saying things like “I am your worst nightmare” to Microsoft execs in elevators at the time.

The first major victory for the concept for “open source” was when Netscape open sourced their browser, starting the Mozilla project. So at that time, Mozilla developed a process oriented towards customization and adding features in a very open and very online way made sense as a foil to Microsoft, and I personally think they gained market share partly based on their critique of Microsoft’s monopoly power and partly because there are a lot of programmers that value extreme customization and have a lot to say about expanding the available options. I think “open source culture” is deeply tied up in theses events. Mozilla eventually formed a financial partnership with a young and hopeful Google that was all about spreading information, and it worked really well!

But Microsoft is not seen as scary any more, and Google isn’t a fun upstart shaking things up anymore.

So Mozilla is left with a partnership with Google, but Google has in-housed as much as possible and copied the cultural processes developed at Mozilla. All with more funding. How do you turn developer discontent (and a critique of monopoly power) into market share under these circumstances? My personal feeling is that Mozilla will be in a tough negotiating position until (1) a coherent critique of consumer tech gains traction among programmers and (2) they can find some other partner or revenue source. I can imagine that critique could be about surveillance, and it’s interesting that some ex-Mozilla leadership created Brave and try to make that case while trying to eat into ad revenue directly with alternate partnerships. I have a feeling that the surveillance has not reached peak levels yet, and I feel like it’ll take many more surveillance controversies before the reputations of the big companies change significantly enough for this all to get going.

Again, I don’t think this is a critique of Mozilla or Rust at all. Knowing the details of TAC, I do feel weird about the level of dominance that Chrome and Android have, but that is in no way a critique of the engineers working on any of that. My expectation is that many Google engineers have never even heard of TAC. I certainly did not know about it.

The point is that the psychological factors you mention seem to be rooted in the history of “open source” and the culture around it. I also think the strengths and weaknesses of particular processes cannot be fully assessed without looking back 20 years later and thinking about the extent to which they have proven to be Microsoft’s “worst nightmare” and what effects this all has had on cultural expectations for projects that are funded other ways.

Conclusion

Elm has been on a “goodwill model” with both Prezi and NoRedInk. “One salary with no strings attached. Do your best!” I do not know of widely used languages that survive on zero full-time workers, but you can do a lot with one worker! I think when you get to three or four, you have a lot more room to balance building relationships, working on technology, interacting online, promoting the project, communicating vision, etc. Once you can cover those aspects, my impression is that additional capacity can end up going towards adding features, so I think higher headcounts than that are important if you want a language/tools with lots of features. (I’d be confident saying that this definitely happens when you get in the 20+ range!)

The main point is that I really think that people underestimate the role of funding/headcount in supporting good working relationships in open source languages. Zero to one is huge. One to two is huge. Two to four is huge again. My feeling is that with languages like Elm or Python or Ruby, some fraction of people who use the languages for work may also build packages or tools. Some fraction of those people may end up building good personal and working relationships with paid workers. Some fraction of those people may carve out some time at their day job to volunteer on the body of work they have produced over time, or maybe to volunteer on high-stress roles that are not covered otherwise. When the open-source work supports your business enough that it can justifiably be done in work hours, my sense is that a person can weather more high-stress events and more high-stress roles. This person is volunteering, but in a way that starts to look more like working on a normal software teams. Work is paid, people know each other and communicate often, etc. There is not membership or paid voting or central funding, but a somewhat traditional software team has formed nonetheless!

A project with higher headcount may just hire for these roles directly, and the long process of building personal and working relationships is skipped by just having job interviews and clearly defined roles. Presto! You have someone who is aligned on project vision in roles that may not be filled for years otherwise! Without that option, it definitely takes longer to fill the roles with higher emotional labor where “nights and weekends” can burn people out really fast. Maybe I am particularly bad at communicating and developing relationships online or asking for help with work that can be much more emotionally difficult than normal software jobs. I am trying things that may help with that, both personally and with the ideas for ESF to focus on community events. My overall experience with online forums has accumulated into expecting to always get slammed, if not today then maybe in a few years. So the positive messages people sent over the last few weeks really meant a lot! I ended up chatting with a lot more people than normal and there have already been some cool projects started from that. I got a feeling that sharing even a messy perspective on these things might lead to positive things, in addition to my conditioned expectations. TBD.

Anyway, my goal here is not to criticize any language/community or to claim that everything is perfect with Elm. I fully expect that I have misconceptions about certain things, easily corrected by people with more direct knowledge. I just tend to find it helpful emotionally to look into history/funding when there is conflict, and this is what I have based on what I know so far.

36 Likes
6

In the past few years I’ve met a couple of core team members of big languages and small, and language creators, and noticed some patterns that emerged somewhat independent of funding.

  1. I keep meeting Go core team members, and there do seem to be a lot of them. They appear to mostly be hyper-focused on one thing: building an education website, squeezing 10% more performance out of the implementation of a cryptographic library, improving GC performance, etc. For what it’s worth, they all seemed to be noticeably happy tool. I can’t speak to programming language development, but ‘big company syndrome” definitely exists, I’ve experienced it first hand many times, and it can slow work down, because coordination costs are very high if you’re not careful.

  2. I haven’t met anyone from Rust, but I’ve spoken to a number of small, young language designers or core team members working on languages with basically no funding, but which had developed Rust’s development and RFC model and each of them praised it enthusiastically. One of these language core team members said it was the single best thing they could have taken from any other language. I take from this that it’s not just a matter of size or funding, process structure matters too.

  3. I met Matz who created Ruby back in February, and he was as over-the-top nice as is advertised. He remarked on his gratitude for those contributing back to the language and how improvement can be driven by the community in this video: https://youtu.be/9TB_EM9IfFo

Elm follows a different development process than these languages. For an open source project of its size, it’s an unusually closed and unusually opaque process. Some of the problems that lead to tensions in the Elm community obviously stem for this.

That’s not to say it’s a bad or worse process than others, but it creates a different structure and that is going to influence people’s behavior.

Perhaps Go level funding would be good for Elm. Elm has similarities to Go, it aims to keep things simple by choosing not to include things, and it’s centrally driven by people who’ve put a lot of thought into the design of such a language. Maybe having that level of funding allows them to hire people to work just on bugs, just on documentation, just on one aspect of community, just on tuning the performance of one tiny corner of the library.

But it strikes me that in the places where Elm allows easy community contribution, Elm has done very, very well, and it’s been around these outlets that community has grown. A few examples:

  1. Third party packages - is there anyone in this community that hasn’t heard of Elm UI? Or Elm-GraphQL? Or Elm-Live? I believe Elm UI and GraphQL are among the most active slack channels, and these packages have organically grown sub-communities within Elm that are great.

  2. Community - there are four major elm conferences, and they are excellent and foster a lot of community. There are also a vibrant Elm Meetup culture, organized mostly by the community.

  3. Education - There are tremendous education resources in the Elm community, some of which come from the language itself, some come from core team members acting outside of that work, but many come from just normal people in this community. We also have an active slack where people spend a lot of time helping each other work through their development in a pleasant, helpful way.

My observation about these community contributions is that they are among the more pleasant parts of this language’s ecosystem. A lot of emotional labor went into each of those things, but I don’t think anyone involved regrets it or feels like it destroys their passion for the project, even when it is hard. It seems often to energize these contributors.

It’s not an easy thing to do, but perhaps in the absence of Google-level funding, the core team could take some small experiments to involve the community in core development more. Evan, you’ve talked about an experiment you’re considering around bug fixes, for example, that’s along those lines. If that’s a success, perhaps you can find other ways to involve this wonderful community you’ve built more. My intuition is that it can be done and will make the whole thing less stressful for you.

13 Likes
7

Could it also be attributed to the headcount? Communication takes a lot of time.

1 Like
8

Maybe, but other languages with far smaller numbers of contributors follow much more open models successfully, and some larger ones do too.

That isn’t advocacy for changing that, it’s me saying that I think it has less to do with headcount and slightly more to do with A) the optimizing goals of the project and B) the preferred ways of working of said team.

1 Like
9

My memory of why I chose Mozilla in this era is that IE was a mess that served me constantly with pop-ups.

Mozilla at that time was far worse than modern browsers (they all were), but I switched a lot of my barely-able-to-use-a-computer family members over to Mozilla from IE and they much preferred it; it just worked better for mainstream users. Eventually, Safari was better still and then chrome came out and I convinced most of them to switch to one of those.

Edit: I’m referring to the Firefox era here, early to
Mid-aughts. Firefox really was a breath of fresh air.

2 Likes
10

I read this all through, and it hangs together quite well. I think whatever you say in public will always have its detractors once your audience grows outside of the comfort zone.

There is an equivalence between Work and Money. In some sense Money is just Work in a static form. You can Work to get Money. You can use Money to buy Work and become a more powerful worker.

Through this lens, the people who contribute to Elm now in a voluntary capacity can also be seen as early investors. We don’t have lots of money to throw at it, so we work instead. But that work is not necessarily being done out of pure benevolence - there is an economic aspect to it. If I invest in Elm, Elm will grow and my investment will grow with it. Even the person who is “just learning Elm for fun”, is investing in their skills, taking a small stake on Elm becoming a popular way to program.

I think you are right that funding for Elm needs to come from sources that are aligned to your aims for Elm. A big no-strings hand-out from a huge internet company would be nice, and you should grab it quick if you can. On the other hand, why would anyone give Elm money if they cannot undertand the investment they are making in it?

The investors that are here already found a great development experience. If you want to expand that investor base, ask the question - what is the great experience they are looking for?

Asside from looking at big money from top internet companies, I think you can also look at growing the existing investment in Elm.

  • At the individual level - attracting more users and contributors.

  • Above that, are individuals or small companies able and willing to donate? Crowd funding each project would be chore as you point out. How about recommending some regular monthly contribution levels for individuals, working consultants, small businesses, larger businesses? It would be a trickle at first, but could grow significantly if it works.

  • Big companies. I don’t have any great insights here - I think those that can afford it are more likely to create their own projects, since we all love to do that. Elm seems more likely to increase its market share from the bottom up, like python did, when developers in the know prove their worth.

10 Likes
11

I think there should be some patron funding set like for Godot https://www.patreon.com/godotengine

3 Likes
12

I can’t find anything to disagree with in what you’ve written here. I think its a fair and even-handed analysis of Mozilla’s situation and of the other companies that are funding PL work right now.

In your first post, it seemed like your comment about JavaScript and the |> operator suggested that languages that take adtech money have less authorial independence and are more beholden to satisfy the loud voices in their community. I can see an argument here: funding these languages is in large part a brand exercise for their funders (I think this is definitely true), community disputes are bad for the brand, these languages have less legitimacy in some way because of how their funding works, as a result of all this the projects have to satisfy users to avoid conflict that would tarnish their ability to recieve funding. I think this line of reasoning would make sense, but in my experience it isn’t really true. People who write blog posts about why a language is bad have very little influence on people who make funding decisions. Go, especially, stands out as a language that is well funded despite getting lots of flak from the commentariat for its very independent design team.

I think you hit the nail on the head though with the comment that the financial structure of the tech industry (nearly all of the money ultimately comes from the big advertisement machine) determines what kind of PL projects get funding. Speaking just as an individual, the tricky bit from my perspective is finding an alignment between your own interests as a technologist and what can actually be funded. I want to work on technology that empowers other people to create; ideally, I’d like to work on end user programming technology that could give ordinary computer users more authorial and curatorial control over their computing. But there’s almost no money in that sort of work, so I’ve so far “compromised” by working on a technology which empowers professional programmers to work on systems software that previously was very inaccessible. There’s a lot of money in having more systems programmers, implementing systems of varying moral valence, and so I can align my interest in building empowering tech with the interest of the capital holding class.

EDIT: I also want to resoundingly agree with the idea that the 4 FTEs is orders of magnitude more funding than 1! The value of headcount grows logarithmically.

7 Likes
13

Not a PL but I’ve always admired Linux Mint’s monthly updates on their blog, with both project news and a long list of donors regardless of donation size. Example: Monthly News – April 2020 – The Linux Mint Blog

They also seem very open about their organization: Team - Linux Mint

4 Likes
14

Maybe one way for the community to donate for elm development of libraries like elm-ui, or elm tools for IDEs or Editors (like for vscode), or even the elm-compiler itself could be done via Github Sponsorship ?

1 Like
15

I found this discussion very interesting, for a number of reasons. I think it is very good that the leadership is asking these questions and pondering how to move forward. To that end, I offer up my two cents which may help broaden the perspective and possibly enhance the discussion.

First, I have to give you some personal background, so that you understand my perspective. I started my career in corporate programming in the early 1980’s. I was an electronics enthusiast and a computer nerd when Commodore, Apple, and IBM PC were the standard of the day. I am not a 10x programmer, like the core community of Elm is, which is why my career path evolved from programming to team lead, business analysis, project management, and consulting. During the last 40 years, I’ve seen technology, methodology, languages, and corporations come and go.

What I have observed is that technical people think in terms of logic and technology. But the world runs on emotion and money - that is - people make decisions on emotional not logic, and money makes the world operate not the best technology.

Companies that are currently funding Elm development are doing it purely for business reasons - they make more money by using the technology than they spend on supporting the development of the technology. The technology is the means to a business objective (earn more revenue than expenses).

From a business perspective, I see the following issues:

Vision

What is the long-term goal for Elm? Every other decision and action is guided by the vision. Do you want it to be a niche language that has a very small but focused user base, or do you want it to be the de facto standard for web development? There are always six ways from Sunday to do anything, but the starting point is always knowing what you want to accomplish.

Product Development

The core product, processes, and 3rd party tools/support has evolved and is in good shape. The evolution of Elm is dependent on the vision for the future.

Governance

There are many models for open source development. Considerations are: how much you want it to be able to scale and how much control you want to retain/relinquish. The current challenges surrounding governance are related to the processes/procedures for decisions and involvement of others.

Sales

Every business needs to earn revenue in order to survive. Money is required to develop and deliver value to customers. The big question is, who are your customers?

Value Proposition

What is the product and/or service that is being delivered to customers? Is it meeting their needs or are there improvements/changes required to better serve the customers?

Marketing

This is different than Sales. Sales is about getting money from customers. Marketing is about building the awareness in prospects that they need your products/services. Right now it is word-of-mouth among the community; money is required to go beyond that.

Administration

Any growth in an organization requires a team to handle the mundane and necessary paperwork, tracking, reporting, and management support.

Human Resources

With open source, there are a number of different levels of participation and stakeholders, for example: the core team, package developers/maintainers, and end-users. Each different group requires a different strategy for their care and feeding.

This is a big, broad swath of inter-related business issues. It may seem overwhelming, but each of these issues will have to be tackled at some point. It all starts with the vision and where Elm should be in 5 years.

One of the challenges that I see for Elm is identified by Geoffrey Moore in Crossing the Chasm. Elm has been adopted by a small group of early adopters. The “chasm” is getting it adopted and used by a larger constituent of customers.

11 Likes
16

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.