Summary
The Non-Profit FOSS Institute (NPFI) provided Steven Huss-Lederman with a grant for the 2018-19 academic year to
implement an HFOSS project in a senior capstone course that spanned the entire academic year. This course
differed from the CO-FOSS model in several ways and thereby offered an opportunity to analyze how this model
might be applicable in fostering HFOSS projects at academic institutions. This report to NPFI summarizes the
techniques used, analysis of outcomes and ideas/steps for using this model.
Overview
This report contains the following sections:
- Overview of Open Energy Dashboard Project
- Overview of Course and Beloit College
- Overview of Model
- Tradeoffs & Experiences
- Why Use This Model
- Timeline of Actions
- Course Outcomes
- Course Materials
Many of the thoughts in this report mirror what others have written about using open source projects in a course.
Thus, the discussion is not necessarily unique but is designed to show how these ideas apply to this model.
Overview of Open Energy Dashboard Project
The Open Energy Dashboard (OED) is a
Humanitarian Free Open
Source
Project (HFOSS) aimed at supporting sustainability by allowing an organization to acquire, store, analyze and
display energy usage. The user interface
is via a web browser where significant code runs on a back-end server. The software is designed to be usable and
customizable by each organization to allow for easy portability. The OED project has existed since the fall of
2016 and, as of spring 2019, is in use in about half a dozen colleges and universities. While it can be used by
any organization, higher education institutions have been the users to date.
An important aspect in terms of its use in a course is that it was begun by a college (Beloit College) and
continues to be driven by academic institutions. This means that while the project has important goals in terms
of sustainability, it also wishes to support its use in computer science education. Thus, the OED project offers
mentoring to interested developers, links to resources to learn about its technologies, and tries to organize
its work so it is accessible to a range of potential developers. It also has a tradition of helping support the
advancement of developers for both the advantage of the student and the project.
OED is a modern application that utilizes a large number of technologies and frameworks. These include Javascript
with promises, React, Redux, PostgreSQL and general SQL. The application is designed to rapidly acquire/store
new data and provide graphical displays of information with sub-second response times. These goals mean that
while
functionality is important, it is generally critical that efficient implementations be used. This means the code
is more complex than a less responsive system. The code also needs greater complexity to meet its goals of
portability, testing for reliability, and other goals. This does not mean that it is beyond the reach of student
developers but it does mean that a solid base of computer science knowledge is needed to engage with the OED
project.
It is noted that while OED was used in this course, the general idea is using an existing open source project in
support
of educational goals in a computer science curriculum. Thus, the model should apply to a large number of
potential projects and can even start as a new project for the initial offering of a course using this model.
Overview of Course and Beloit College
The Open Energy Dashboard project was used as part of the Database Capstone I & II course at Beloit College. This
pair of courses runs over two semesters in one academic year where it generally meets three times per week for 65
minutes for each class. Students normally take both semesters with
limited exceptions for student who are not in residence in a given semester (such as during study abroad). This
report is about the 2018-19 academic year when 17 students were enrolled in the course and four students only
did one semester (two first semester and two second semester so enrollment was constant at 17).
The course has a number of objectives that directly relate to the use of OSS:
- Learning databases. This course must teach students the fundamentals and practical usage of
databases. Some students get experience with certain parts of databases via their work on OED.
However, many would not due to the extensive front-end parts and the fact that significant parts of the
back-end database code is complete. Thus, the course exposed students to the database aspect by having
them create a database to analyze data for the local government. This part of the course was separate
from the OED work and not directly related to this model.
- Learning real-world development skills. Computer Science students at Beloit College take a
separate software engineering course. However, this course is designed to let them apply practical
development skills to a real problem. As such, the students are exposed to coding including learning new
languages/technologies, development environments and shared repositories. The OED project naturally
allows for all of these goals.
- Real-world experience. Beloit College has a Liberal Arts in Practice (LAP) requirement where
every student must do significant work outside the traditional classroom. By working with the OED
project, students get exactly such an experience. Furthermore, doing a humanitarian project fits well
with the goals of Beloit College and the interest of many of our students. This model does not require
the use of a humanitarian project.
- Teamwork. The Beloit College Computer Science Program is committed to exposing students to
teamwork and making them effective members of teams. One reason all students in the course worked on the
OED project was to get them involved in a large team effort that was accomplished through a series of
smaller sub-teams.
- Mentoring. This course sequence deliberately mixes 3rd and 4th year students. The goal is to have
more advanced students mentoring less advanced students. While this often means 4th year students are
mentoring 3rd year students, there are many times where a 3rd year student is more knowledgeable in an
area so is mentoring seemingly more advanced students. This is exactly what is desired so that students
consciously think about both sides of the mentoring relationship: being a mentor and being a mentee. It
should be noted that this model of mixed year students is also due to curricular and staffing
considerations and is not essential to this model.
Some of the setup of this course directly relates to its use for supporting an HFOSS project and were
desirable in supporting this model. These were:
- Grading. As mentioned above, the course had multiple objectives. The percentage value of
different aspects of the course were designed to reflect these objectives. They were (first/second
semester):
- 10%/10% for class participation
- 25%/18% for assignments
- 40%/47% for project
- 25%/25% for examinations
- Weekly wiki. To keep track of all the students in the course, each student has to input a
wiki entry by the end of each week. This was essential to know about how each student was doing
along
with understanding progress on the project. By early the next week, the instructor would review each
entry and put comments in the wiki for the student. The comments included if the work was
satisfactory, suggestions on issues encountered, and, sometimes, a request for a meeting with the
student or team. The weekly entries requested the student to address each of these items where only
work related to the project was included:
- What are the items you accomplished?
- Did you work with others (by item if it varied)?
- How much time did you spend on each item?
- How were you a good mentor/mentee?
- How is it going with your team?
- Do you need or want something to help you participate in the project?
- List what you did for any times that class time was made available as a separate entry from
other work with the time spent listed separately.
- Do you have any questions or thoughts?
- Anything else you think is important.
- Project discussion. Whenever possible, class time was provided on Fridays to discuss the
project. It began with a whole class discussion of general project progress and global issues. It
was followed by the instructor meeting with teams/individuals as appropriate.
These were found to be invaluable as big picture items along with some items students were loath to
put in the wiki entry were brought up.
Overview of Model
An important model for supporting HFOSS in higher education is the CO-FOSS model. The techniques used in the
current course has similarities and differences from the CO-FOSS model as outlined below:
- Existing versus homegrown project. This is probably the most important difference. The current
course
continued development of the Open
Energy Dashboard
(OED) that is an ongoing HFOSS project to develop a freely available energy dashboard. This is a large
and
complex project involving multiple technologies that had been ongoing for two years and is expected to
continue development for many years to come. This contrasts with the CO-FOSS model where the team works
with
a client to design and develop a solution in the time frame of the course. It also means that the final
work
is integration into the project's repository rather than delivery to the client for final deployment.
Thus,
unlike CO-FOSS, there was not a need to find a source of ongoing support as the overall project provides
this framework.
- Large community versus single/limited client. OED does not have a single client but is designed
to be flexible and customizable for use by any organization. As a result, project requirements come from
listening to the larger community and filtering those needs/requests into general features for future
project
releases. This contrasts with the CO-FOSS model where a single entity provides the requirements and
there is
one
person or a limited number of people who discuss requirements and review progress. Combining with the
fact
that
OED is an ongoing project, the students experienced the design process in a very different way than the
CO-FOSS
model. With OED, the students were working with requests/issues that already existed within the project.
Therefore, they did not do a complete system design but worked within the existing design with much more
limited
scope. Furthermore, the students did not tend to interact directly with clients but did so indirectly by
working
with the OED project manager (also the teacher of this course) to understand and refine requirements.
This
does
not mean that a client-oriented design focus was lost but it does mean that students did not experience
the
creation of a new project and did not have the traditional client of the CO-FOSS model. It should be
noted
that
somewhat over half the students in this course had already completed a software engineering course so
they
already had
the more traditional exposure in that course.
- Learning project technologies verses picking them. Since OED already existed as a modest size
project, the main technologies had already been selected. These included JavaScript using promises,
React,
Redux & PostgreSQL and many others that had more limited usage. Depending on what aspects of the OED
project
were being worked on by a student, they needed to learn a number of these technologies well and have a
basic
understanding of the others. This required allocating many weeks of the course to learn what
was needed. This contrasts with the CO-FOSS model where the project can pick appropriate technologies to
meet the user requirements and also take into account student knowledge/ability to learn. The CO-FOSS
projects that have been discussed to date required fewer and simpler technologies. This means that
having
students jump into the OED project was much more complex and this has been noted by others who have
worked
on existing OSS projects.
- Learning verses designing an architecture. OED is a non-trivial project with an existing
architecture. This necessitated students learning the OED software design and understanding the
interrelationships of parts of the code. As with most existing projects, but esp. as one that is only a
few
years old, there were quarks and deficiencies in the design of OED. This made this process even more
complex.
This contrasts with CO-FOSS where a new project is begun or a modest size project is reworked. With OED
the
students had much more expose learning to understand existing code than CO-FOSS where they get a
strong
overall design experience. The OED students needed to learn how to be "productively lost" (a phrase in
the
OSS
community) where they balanced a complete and detailed understanding and what was needed to get the job
done
in
a reasonable period of time. The OED project, in general, uses more real-world technologies that are
utilized in
larger projects outside of academics and students appreciated that applicability. On the other had, the
instructor had to monitor and deal with student frustration in feeling overwhelmed by what needed to be
understood.
- Student goals and timelines.The goal was for each student to contribute code that was integrated
into
the OED project. This might happen multiple times over the course and could happen at any time. Also,
given
the size of certain requirements and uncertainty of the scope of development, students might not
complete
the work and leave a fork of partly developed code. Clearly this was less desirable but a reality, esp.
given a wide range of student abilities. Students could also complete coding until near the end of the
course. (This did raise issues when changes were needed with no time to complete but this did not mean
the
work was lost or could not be used.) In the CO-FOSS model, the final goal is a working application so
success is defined differently. Also, the application needs to be given to the client with documentation
so
time is left at the end of the course to work with the client. This leads to somewhat different student
experiences where, in hindsight, the OED experience would have been enhanced by stronger deadlines.
- Year-long verses semester project. Anyone working with students to create software in an academic
course appreciates the fact that semester (or quarter) courses impose significant constraints on
the
work done by students. The current course runs as ½ a standard course per semester for two
semesters.
Having a longer time frame makes it much easier to deal with software issues and questions. It also
allows
for larger chunks of work to be accomplished. The break between semesters does represent a scheduling
challenge as the gap allows students to forget what they were doing if a task remains incomplete. In
contrast, the CO-FOSS model has been run in a traditional one-semester course. The issues of this time
constraint has been discussed by CO-FOSS users. The difference of one vs two semesters does not seem
fundamental to either model was they both could be done on different timescales. However, the fact that
there was a difference in the courses to date does mean that the experiences were influenced by this.
Tradeoffs & Experiences
Several of the tradeoffs and experience have been mentioned in the previous sections. Here greater details
are
given with an honest assessment.
- Development environments. As others have noted, getting students working within the development
environment of a OSS project can be a daunting task. OED tried to address this issue by investing in
Docker setups to allow for more automated creation of development (and production) environments. A
significant issue encountered was Microsoft's decision to discontinue support for Docker on
platforms other than Windows Professional. At least ¾ of the students used Windows but none had a
professional grade installation of the OS. This necessitated doing manual installs that were more
difficult, error prone and required help from more experienced developers on the team. The time spent
and aggravation experienced by students was a distraction to the real goals of the course and project.
This was quickly realized and a full development environment (under Linux) was created in the student
lab. Unfortunately, students continued to spend many hours trying to get OED working on their own
Windows machines rather than use the one set up in a lab. In hindsight, the course should have made it
explicit that only Linux-based systems were supported (including Mac OS) and students without this OS
needed to use a provided machine or a containerized/dual boot system. This would have saved student
frustration as well as a drain on the time of more experienced developers. It seems a good practice to
consider not just the ease of installing the development environment but the range of machines that
students
can use.
Another issue encountered was the command prompt. Many students in this course had no or limited
experience in working with the command line. Even though directions included explicit commands to
run,
students had issues and were unable to overcome seemingly simple issues. This situation was
magnified on
Windows machines where manual installs via the command line were needed. In hindsight, the course
should
have had an early lab on command line usage to avoid many of these problems.
Finally, a few students had old machines or ones that were in a poor state in terms of installed
software. First, this lead to operations taking much longer than should have been the case. Second,
students would sometimes skip installing a software package because they found it was already
installed.
However, if the installed version was old, the student would encounter errors that were difficult to
understand. Again, this was more of an issue on Windows machines where the Docker install should
have
taken care of these problems. (Note it does not fix all issues as the development IDE used might not
support
recent
versions of the languages being used.) There is no trivial fix for this situation but students were
given information on the minimum/desired versions of software. Finally, if students had used the
provided lab machine then this would not have been an issue but they did not wish to do that.
- Learning technologies. As noted in the overview of the model, OED uses a
number of
modern technologies/frameworks. These often went beyond what students had already experienced in
previous courses. This was a known concern and students were provided with online links to overview,
learning and reference materials for all the main technologies. Furthermore, they were given three weeks
of class time to learn in a self paced structure where questions could easily be asked.
In several cases, enough students had questions about an area that a mini-lecture was given on that
topic. It is frustrating to report that this did not work out well. About ⅓ of the students who
were self motivated and/or had more experience took advantage of this time to get a basic understanding
of the OED technologies. The remaining students did not with most not seeking any help. As would be
expected, these students had ongoing issues with the project. It is clear that the issue was not a lack
of
course time dedicated to the process but how it was done. In hindsight, the course would have benefitted
from these steps:
- Regular mini-lectures to introduce a topic. This would serve as a chance to frame the needed
technology and made it easier for students to ask questions.
- Short quizzes on a frequent basis. This would have reminded students were they should be in the
process and given the instructor more timely feedback. The instructor mistakenly thought that
students in the capstone were mature enough to self-regulate and doing this was an important
capability in preparation for working outside of classes. While this probably should have been
the
case, it was not and external motivation was needed.
- Every student should have been made to do a simple pull request to the project such as a change
in
wording to documentation. This would have helped guaranteed they had their development
environments
set up. Possibly having to specify the change on the development branch or some other activity
might
have help. The trick is something that would be hard to do without a true development
environment
(something harder due to the nice interface from GitHub). Some students tried to work without
this,
such as used a text editor, and it naturally lead to significant issues.
- It may also have made sense to require students to turn in short program utilizing the needed
technologies.
When the depth issue was recognized half way through the course, the students with issues were
required to go through sections of the code to add/modify the documentation. The goal was to force
them to engage with the technologies. This was done by setting up small teams (2-3 students) where
some teams could interact due to overlap in the technologies being engaged. The OED project was
broken up in two dimensions: technologies & features where there were three main areas for each
of these. This lead to a grid of 9 items where some overlapped so significantly that they were
combined. This ultimately lead to 5 areas of technologies/features. The twelve students who needed
this engagement were assigned to one of the areas and a few were assigned to a primary and secondary
area. Each area was also assigned a mentor who had a good understanding of the area (from the
⅓ of students who had a good understanding of the project). The teams then worked on their
area. Once documentation was done, a team working on the same
technology and/or feature was notified so they could review the new documentation. This served to
both check the proposed documentation and force students in one team to stretch into related
areas. After this step, teams submitted a pull request to get the updated documentation included
into the OED project. Each team was also to include an overview of their area in a master document.
Students uniformly liked this activity and having done it earlier would have been much better. The
reality is that about half the students (7 of 12) engaged successfully in the sense that they were
then
able to engage with coding activities in the OED project. The other half continued to struggle but
almost all of those students did not reasonably engage in this process.
It is noted that it takes a good technical understanding of a project to separate out the
directories/files into different boxes. For reference, OED was separated as follows:
|
|
Features |
|
|
graphics |
Data acquisition |
Admin
|
Technologies |
DB |
team 5 |
|
team 4 |
API/server |
|
team 2 |
Front-end |
team 1 |
team 3 |
- Background knowledge. With about half the students being 4th year having taken a software
engineering course and the other half being 3rd year without software engineering, there was a greater
range of background. As previously mentioned, this is partly to allow for exposure to mentoring. While
there are advantages, it was certainly trickier to run a large project with this many students when some
did not appreciate software engineering and had limited experience in working on a large software
effort. It was clear that the less advanced students had greater difficulties in effectively working on
the project. If possible, it would be best if students involved in a larger, real-world project already
had experience in developing at least a medium size project and working on teams.
- Tracking student progress. It can be trick to know how much time and effort is needed to address
an issue or
feature in a project. As a result, it was often difficult to know if a student or team was applying
sufficient time and being effective. The weekly wiki helped but it was still difficult. Augmenting these
with regular meetings, probably being once a week would be ideal, would have helped in this matter. A
combination of group and individual meetings would likely be best. The reason these were not done was a
time issue when nearly 20 students were involved. However, in hindsight, they should have happened on a
regular basis and not been more at the request of students.
- Deadlines. A modest fraction of the students did not make sufficient progress. Having clearer
deadlines would have helped. Being more rigorous in the agile deadlines where everyone did a sprint for
a specified period (say two weeks) would have helped. As long as students understood that they did not
have to finish in one sprint but needed to discuss why not and the plan for completion then it would
probably work. It was difficult to be reasonable and flexible in deadlines while still holding students
accountable in work where the difficultly of work was tricky to judge in advance.
An important consideration in missed deadlines is the impact on other students and the project.
Balancing the educational needs with project goals was a major concern throughout the course. At
times it required shifting students off a team or adding a team member so progress was made. As
skills and effort for each student became clearer, the more capable students were placed on more
critical tasks and others were placed on desirable tasks that would not impede the project if
deadlines were missed. This balancing act took the instructor more time than desired but no easier
alternative was found.
- Evaluation. The instructor mistakenly thought that the feedback provided on each weekly wiki
along with periodic discussions would be sufficient for the students to know their standing given this
was an advanced course. Student feedback clearly indicated otherwise, esp. for students who were having
issues. Thus, in the second semester a clearer set of criteria were provided and more regular feedback
given. The project was worth almost 50% of the course grade. The criteria for the second semester were:
- 15% for wiki and check-ins. Students received written feedback on each wiki entry and verbal
feedback at meetings.
- 10% for quality of community membership (helping others, working well with others, meeting
deadlines). Students were instructed to put information about this in their weekly wiki. The input
of the student and other students in the course were used to determine the situation. Entries in
project spaces such as comments on pull requests counted toward this area.
- 50% for student efforts. This was based on weekly wiki entries that required accounting of time
spent on each task and the entries of other students. The number and size of pull requests or code
in a branch was also used. A discussion of this area was an important part of the final student
meeting.
- 25% for the quality of student work. This was primarily based on code in the project repository
where care was taken for team work where the id associated with a pull request might be another
student. The wiki entries and discussions helped with this but students were also informed that all
team members should be doing commits at points in time. This was an important part of the final
student meeting where the instructor explicitly talked to other team members to get their take on
the work of other students.
It was found that a non-trivial percentage of students had issues in getting in a substantial pull
request to the project. This is why such a large percentage was allocated to effort compared to
quality of work. The community membership was important to acknowledge the importance for the
project and others in working as a team.
Why Use This Model
The model described was chosen because it was a good fit for the objectives of the course. Here are a few
thoughts on when and how this model might be appropriate for others.
- If you want to give your students a more open and potentially shifting experience then using an existing
open source project can help. Students need to learn the tools and technologies of the project and these
often mirror those of the real-world developer. Students are also faced with a large code base so they
learn how to jump in and move forward without understand or learning about all the parts.
- Knowing the open source project is helpful (but not necessary). If the department/institution,
instructor or students have experience with a software project then this is a clearer avenue to its uses
in a course. It is still important to be sure it will be viable for your course objectives. If you don't
have an existing project, picking one that aligns with student or instructor interest can help with
motivation and general knowledge of the application area.
- Having a project that is used more generally within a department means you build on the use outside your
course. It also gives students an avenue to engage with the project after (and even before) your course.
For a number of students, the deepest knowledge and experiences happened after the first offering of
this course when they continued work as a special project. Building on their existing knowledge gave
them a very rich experience. In addition, having a range of students working on and knowing about the
project on an ongoing basis created an ecosystem where students could help each other and felt part of
an important and larger effort.
- One issue with an extant project is engaging students with more limited experiences. In the case of OED,
having students do documentation, giving them class time to learn the needed technologies and having
tasks that were easy for a beginner all helped with this issue. This issue can be magnified when you try
to integrate in multiple ways within a department. Thus, knowing a project can support your range of
students is important.
Timeline of Actions
Similar to doing any large project, utilizing an open source project means doing items at appropriate time.
Here
are some of the more important ones for this model.
- The biggest choice is deciding on an open source project. As written by others, you need to consider the
size, scope, technologies, friendliness, etc. of the project. Some ideas on considerations for the
project are elsewhere in
this report. Starting early cannot be stressed too much. It takes time to evaluate projects and one
often finds that a promising one later turns out to have issues. Thus, starting at the high-level,
narrowing the field, and then looking more closely can be a valuable way to go about this. If at all
possible, starting months in advance is the way to go.
- You want to begin interfacing with the project early in the process. Initially it is to understand how
welcoming the people are to students, how do they deal with tasks including submitting proposed code,
and do they answer questions in a timely fashion. Later on you need to see the documentation for the
project and technologies, the tasks that are available and how to determine appropriate ones for your
course. In an ideal situation, you can get contacts/mentors for your students and the project will
listen to the timeline/goals of your course so you know everyone is on the same page. The ease of
working with
a friendly project that offers support cannot be underestimated. Having this in place at least a month
before your course begins will make the process go much smoother.
- If you want to expose your students to the clients involved and it is possible for the chosen project,
then you need to contact people in advance of the course to make arrangements.
- Before the course begins you need to have a list of appropriate tasks where they are ideally separated
into types (newbie, modifying code, new code, design, etc.). In some cases the project may not clearly
articulate the larger picture of why this feature is needed. Getting that information so students can
appreciate the work is both valuable and motivating. In the case of OED, the GitHub issues page was
useful but insufficient. The project organizer provided more information and this was then used to
create a course specific document on the features that would be addressed, why they were being
addressed, and links to needed resources to engage with these tasks. An ideal project might have this
but generally they do not.
- Once the semester begins you will need to form teams to work on the project. There are different ways to
do this: student interest, grouping students with similar or a range of experience, randomly, rotating,
etc. It isn't clear that one way is superior but having a plan and being flexible is important. With the
uncertainty of engaging in an ongoing project, this flexibility to shift students around is often
critical as the situation unfolds. For the OED experience, it was important to get the teams set up
early so students clearly understood their work and had other students to work on issues they
encountered. Generally, lone students tend to spin their wheels more and flounder.
- Keeping track of what is going on with individuals, teams and the overall project is important. As
discussed elsewhere, students did a weekly wiki as input. Also, every Friday was a day for team reports,
to raise issues and track the overall project. The OED experience over the years has lead to more rather
than less oversight. It was found that fewer students ran into issues and the project was more
successful with greater oversight. This was even the case with better/motivated students who worked on
OED in special projects.
- Having a timeline for work to be completed and track the progress. Students often don't appreciate
the time needed to accomplish tasks and when they are starting to get into trouble. The tracking just
mentioned should help the instructor appreciate the situation. However, students should get a rough
schedule at the beginning and it should be revised, as appropriate, throughout the course. In the OED
experience, students who had issues had a tendency to give up and needed redirection. On the other end
of the spectrum, students who experienced few issues sometimes slacked when they should have begun
focusing on new tasks. Only though oversight by the instructor were these issues addressed.
Course Outcomes
Overall, the course evaluations showed that students had a good experience. They appreciated being exposed to
real-world software and development. As seen by others, they were motivated and excited to work on a
humanitarian project. Another important outcome is that a significant fraction of the students report that
their work on OED was discussed during job/internship interviews and/or they felt it was important in
securing a position. It is also noteworthy that this work/project was noticed by the institution and its
leaders.
Course Materials
Course materials that might be of value to others are: