Skip to content

Software Development News: .NET, Java, PHP, Ruby, Agile, Databases, SOA, JavaScript, Open Source

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Google Open Source Blog
Syndicate content
News about Google's Open Source projects and programs.
Updated: 1 hour 7 min ago

How to format Python code without really trying

Mon, 03/30/2015 - 17:00
Years of writing and maintaining Python code have taught us the value of automated tools for code formatting, but the existing ones didn’t quite do what we wanted. In the best traditions of the open source community, it was time to write yet another Python formatter.
YAPF takes a different approach to formatting Python code: it reformats the entire program, not just individual lines or constructs that violate a style guide rule. The ultimate goal is to let  engineers focus on the bigger picture and not worry about the formatting. The end result should look the same as if an engineer had worried about the formatting.
You can run YAPF on the entire program or just a part of the program. It’s also possible to flag certain parts of a program which YAPF shouldn’t alter, which is useful for generated files or sections with large literals.
Consider this horribly-formatted code:
x = {  'a':37,'b':42,
y = 'hello ''world'z = 'hello '+'world'a = 'hello {}'.format('world')class foo  (     object  ):  def f    (self   ):    return       \37*-+2  def g(self, x,y=42):      return ydef f  (   a ) :  return      37+-+a[42-x :  y**3]
YAPF reformats this into something much more consistent and readable:
x = {'a': 37, 'b': 42, 'c': 927}
y = 'hello ' 'world'z = 'hello ' + 'world'a = 'hello {}'.format('world')

class foo(object):    def f(self):        return 37 * -+2
   def g(self, x, y=42):        return y

def f(a):    return 37 + -+a[42 - x:y ** 3]
Head to YAPF's GitHub page for more information on how to use it, and take a look at YAPF’s own source code to see a much larger example of the output it produces.
by Bill Wendling, YouTube Code Health Team
Categories: Open Source

Google Code-in 2014 wrap up with OpenMRS

Fri, 03/27/2015 - 17:00
OpenMRS is a medical records system used around the world, especially in places where resources are scarce. It’s also being used with Google’s chlorine-submersible tablets designed for Médecins Sans Frontières to use while treating ebola patients. The OpenMRS community recently participated in Google Code-in, providing young students with an opportunity to get involved with real open source projects and learn about contributing to them. Chaitya Shah, one of OpenMRS’ two grand prize winners, shared this story with us about his participation in the contest.
For 7 weeks in December 2014 and January 2015, I worked with OpenMRS in the Google Code-in (GCI) competition. GCI introduces highschool aged kids to open source software development by providing a wide variety of tasks we can complete. For me, it has worked wonders. I’d been interested in the concept of open source software for about a year and even participated in GCI 2013, but this year, the experience turned my interest into a passion. I worked on many new things, met lots of new people, and learned several important skills along the way.

A few days before the competition started, I decided to see how OpenMRS’s software worked. I went through the GitHub repositories and tried to get openmrs-core, the main application, running. After a few tries and the help of several contributors on IRC, I was finally able to do so. Their help showed me what the OpenMRS community was truly about: everyone was very helpful throughout the contest and there was always someone online to help me out at any time of the day.
Several of the tasks I worked on this year were much more complex than the ones I worked on last year, giving me more of a challenge and motivating me to put forth my best effort! The early tasks, however, involved getting acquainted with the OpenMRS community and learning how things work in the organization. Several of these tasks taught some key aspects of open source software or of programming in general. One of the simplest but most important tasks was introducing myself to the community. If the communication between a developer and an organization is weak, the code produced will suffer. It was also inspiring to see so many other people interested in contributing to OpenMRS through GCI.
After learning the basics of OpenMRS, I started to explore tasks in the UI Revamp epic. With guidance from a mentor, I worked on making the OpenMRS ID site look more like the redesigned wireframes provided. These tasks really taught me a lot about design, one of my weak points. I used to know very little about HTML/CSS in general. The revamp tasks taught me about good practices in UI Design and I loved every minute of it.
In the last two weeks of the competition, I decided that I was ready to contribute something brand new to the organization. While deploying OpenMRS on the OpenShift cloud platform as part of a task, I found the developer guide was vague in some areas and difficult to follow. It took me a few days and some experimentation to get it working. To ensure that others wouldn’t have the same troubles, I made two videos showing the exact steps to follow: one for Windows and one for Unix-based systems.
After that, I decided to take on a Docker task. Docker is a system that lets you build, ship, and run distributable applications. This task directed me to create an image that downloads, sets up, and runs OpenMRS automatically. I was slightly overwhelmed at first, but Docker proved to be quite useful because it uses a system of containers rather than virtual machines, making it much faster and easier to deploy applications. I felt a big sense of accomplishment once I had finished publishing my work, writing up documentation, and making a quick video tutorial on how to set it up.
I learned a lot from OpenMRS and GCI this year. I was especially impacted by the weight that community interaction has in open source work. Previously, I’d always had the notion that being a programmer is very lonesome, sitting in a room with nothing but a computer for many hours at a time. However, I now know that everything in open source software development is collaborative; everyone works together to accomplish a single goal. I hope to someday find a job with a company that embraces this collaborative nature. Thank you to OpenMRS and GCI for an awesome experience this year!

By Chaitya Shah, GCI grand prize winner
Categories: Open Source

Classp: a “classier” way to parse

Tue, 03/24/2015 - 17:00
If you’ve worked on compilers, translators, or other tools that need to read a programming language, chances are you’ve spent many painful hours detailing that language’s grammar. Recently, we opened up the source code for Classp, a side-project a few of us have been working on that demonstrates it’s possible to have an automatic parser generator that is not based on formal grammars. Instead of grammar productions, you write classes similar to C++ or Java and you write class patterns to define the syntax. Although there are libraries like Boost.Spirit and Scala Parsers that give you a nice way to write a grammar in the programming language itself, in the end you are still writing a grammar. Even though Classp looks a lot like C++ or Java, it is not just a C-like way to write a grammar. It’s an entirely different way to specify syntax.
Grammars are great for diagramming a complex syntactic structure for human readers, but as a computer specification, they leave a lot to be desired. Four key problems with grammars inspired us to work on Classp as an alternative.
First, a grammar is intended to represent the actual syntactic structure of the language: all of the little details like what goes first and what goes second, where to put your commas and semicolons, where can you substitute one thing for another, etc... But this surface structure doesn’t really matter to the programmer who wants to process the language. It just gets in the way. What you really care about are the logical parts of the declaration: what is the type? What is the name? Is there an initializer and what is it?
Second, many common parser generators don’t actually specify any tree at all. They let the programmer write actions to build up a parse tree. But the actions in most systems tend to form an awkward fit with the grammar.
Next, when you write a grammar, you have to worry not just about the surface structure of the language, but also about how the language will be parsed. You have to write the grammar around ambiguities in the language and sometimes around other features. You can’t just write the rules as you would write them for a human reader:
Expr ::= Int | ( Expr ) | Expr + Expr | Expr - Expr | Expr * Expr | Expr / Expr
instead, you have to write them in a way that avoids ambiguity:
Expr ::= Expr + Term | Expr - Term;Term ::= Term * Factor | Term / FactorFactor ::=  Int | ( Expr )
Finally, grammar-based parsers are extremely verbose. For serious parsing tasks, it is common to write a grammar, design a parse tree, write actions in the grammar to create the parse tree, design an abstract syntax tree, and write code to translate the parse tree into an abstract syntax tree. There are many dependencies among these parts that all have to be kept consistent over the life of the program. It’s a complex and error-prone process.
Classp attempts to avoid these problems. The abstract syntax tree is what programmers typically want to work with. With class patterns, you only have two jobs: design the abstract syntax tree and write a formatter for it. (A formatter is the function that writes out the abstract syntax tree in the target language.)
Here’s an example class declaration for an abstract syntax tree. The class pattern is the part inside the parentheses of the syntax statement: “arg1 '+' arg2”.
class Plus: Expression {  Expression arg1;  Expression arg2;  syntax(arg1 '+' arg2);}
This class pattern says that to print a Plus node, you first print arg1, then you print a plus sign, then you print arg2. So it looks like a nice formatting language, but where do we specify the parser? The answer is that we don’t specify a parser; Classp will invert the formatter to generate a parser for us. Since formatters are typically much easier to write and maintain than parsers, it almost feels like magic.
Classp is still a work in progress. We still have to deal with ambiguity in languages, features that are only output in one way but may be input in several ways, and a few other issues. But it’s ready to play with now and we’d love to hear from others interested in this subject. To learn more, visit

By David Gudeman, Classp team
Categories: Open Source

GSoC project Sambamba published in scientific journal

Fri, 03/20/2015 - 17:00
Student applications for this year’s Google Summer of Code (GSoC) are still open until March 27th. One of our goals with GSoC is to inspire young developers to participate in open source development, hopefully continuing well beyond the summer. Pjotr Prins from the Open Bioinformatics Foundation shared this story with us about a GSoC 2012 student who has continued leading the development of a software tool used in laboratories around the world. That tool, Sambamba, was recently featured in an Oxford University Press scientific journal.

The Open Bioinformatics Foundation (OBF) participated in Google Summer of Code (GSoC) in 2012 and again in 2014. One of our projects, Sambamba, enables users to rapidly process large sequence alignment files in the SAM, BAM and CRAM formats using parallel processing. Sambamba, which means “parallel” in Swahili, was recently the subject of a paper published in Bioinformatics Journal by GSoC alumnus Artem Tarasov. Since the tool is now used in DNA sequencing centres around the world, Artem has become well known in the bioinformatics community as Sambamba’s creator.
When we participated in GSoC 2012, we accepted five students, one of whom was Artem. His project was to “write the fastest parallelized BAM parser in D” as an alternative to the existing SAMtools software written in single-threaded C. I consider the D language to be particularly well-suited to bioinformatics given its modern hybrid OOP/functional syntax with close-to-the-metal performance optimizations.
Even before GSoC started that year, Artem was doing research and cranking out code. In his blog, he wrote about learning the D language, dealing with parallel executing code, and the sometimes-buggy compiler and garbage collector. The file formats he was working with are complicated and contain many assumptions, but he made wise choices which led to a very effective piece of software: people tend to rave about Sambamba when they use it the first time. Artem and I continued working on Sambamba after GSoC and before long, I found that he was the one mentoring me!
Since then, Artem has been invited to visit the Cuppen sequencing lab in the Netherlands where he added depth analysis to Sambamba. This is also when we started work on the manuscript for the Bioinformatics Journal. Later, the OBF was able to sponsor a second trip to the European Bioinformatics Institute in Cambridge, UK where he and I took part in a Codefest and met with other bioinformatics researchers and developers, including some OBF contributors.
Artem isn’t our only GSoC student who has continued making a difference in open source. Four of our five GSoC 2012 students are still active FOSS committers on GitHub, with three of them continuing in the bioinformatics space. Although GSoC can be competitive and we haven’t been accepted into the program every year, we’re grateful for the opportunities it has given us. Organizations like OBF and SciRuby are proof that GSoC and scientific projects work really well together. Without GSoC, Artem and I would probably not have ever met. He and I both hope to introduce more students to scientific open source projects in the future.

By Pjotr Prins, Sambamba GSoC Mentor
Categories: Open Source

Google Summer of Code: Meet-up Round-up!

Wed, 03/18/2015 - 19:30
Over the past ten years, Google Summer of Code (GSoC) has given over 8,500 students a bridge into open source communities. GSoC alumni have played a large role in the program’s success by encouraging their fellow students to take part. Around the world, energetic students who have already participated in GSoC lead the way by hosting meet-up events that help others learn about the opportunities GSoC provides and get answers to their questions about the program. Who better than GSoC alumni to tell students what it’s really like spending the summer coding for open source projects?

Student applications for GSoC 2015 are open until March 27th, so the community has been spreading the word at school and beyond during the past few months. Here are just a few of the student-initiated events we’ve heard about recently.
Buea, Cameroon: 17 December 2014
Chris Nuvagda wanted to spread the word about GSoC after finishing his project, so he reached out to other GSoC alumni at the University of Buea to make it happen. Over 150 students attended the meet-up, with about 30% being women -- a number we’re happy to hear since only 10% of GSoC 2014 students were women and we’re trying to improve on that. The 5 presenters spoke about their experiences in GSoC and the benefits of it, gave advice on writing a great proposal, and shared what it’s like for women working in open source. Read more...

Delhi, India: 31 January 2015
Heena Mahour and Ayush Gupta hosted a meet-up for about 200 prospective students and open source enthusiasts. With help from others in their community, they organized a full-day event featuring 8 speakers. Students who previously participated with organizations like KDE, OWASP, and Mifos shared their experiences and answered questions from the audience. Technical topics were also discussed, including memory leaks and an introduction to version control systems. Read more...

Udine, Italy: 24 February 2015
Claudio Desideri spent GSoC 2014 coding for KDE and organized a meet-up for students at the University of Udine. With 80 attendees, there were just barely enough seats for everyone. Claudio shared his personal experiences but also learned that many students who might want to apply to GSoC fear being judged by potential mentors or failing out of the program. He helped reassure students that GSoC is an opportunity to grow and that they aren’t expected to already know everything. Read more...

Bangalore, India: 2 March 2015
With a desire to increase the participation of nearby colleges, especially among female students, Tejas Dharamsi and Rajath Kumar organized a GSoC meet-up at the Google Bangalore office. 102 students, including 44 women, from 11 colleges gathered to hear from the 6 speakers who shared details about the coding projects they worked on and gave practical advice for students applying to GSoC for the first time. The ending panel discussion gave attendees a chance to ask questions and offered students encouragement to apply for the program. Read more...

Kampala, Uganda: 12 March 2015
Organized by Kaweesi Joseph with help from OpenMRS, the organization he worked with during GSoC 2014, more than 110 students gathered for this meet-up. The 5 presenters discussed the benefits of contributing to open source, how students can participate in GSoC, and how OpenMRS is being used in Uganda. Read more...

We thank these GSoC alumni and many more who are spreading their enthusiasm for open source in their local communities. It’s because of local networking events like these that GSoC is able to have rich representation from all around the world. We look forward to welcoming some of these meet-up attendees into the upcoming Google Summer of Code 2015 -- don’t forget to apply before March 27th!

By Ashleigh Rentz, Open Source team, with thanks to everyone who allowed us to share their stories and photos here.
Categories: Open Source

Google Summer of Code now open for student applications

Mon, 03/16/2015 - 20:00
If you’re a university student looking to earn real-world experience this summer, consider writing code for a cool open source project with the Google Summer of Code program.


Students who are accepted into the program will put the skills they have learned in university to good use by working on an actual software project over the summer. Students receive a stipend and are paired with mentors to help address technical questions and concerns throughout the course of the project. With the knowledge and hands-on experience students gain during the summer, they strengthen their future employment opportunities. Best of all, more source code is created and released for the use and benefit of all.

Interested students can submit proposals on the website starting now through Friday, March 27 at 19:00 UTC. Get started by reviewing the ideas pages of the 137 open source projects in this year’s program and decide which projects you’re interested in. Because Google Summer of Code has a limited number of spots for students, writing a great project proposal is essential to being selected to the program — be sure to check out the Student Manual for advice.

For ongoing information throughout the application period and beyond, see the Google Open Source Blog, join our Summer of Code mailing lists or join us on Internet relay chat at #gsoc on Freenode.

Good luck to all the open source coders out there, and remember to submit your proposals early — you only have until March 27 to apply!
By Carol Smith, Google Open Source team
Categories: Open Source

Google Code-in 2014 wrap up with Haiku

Fri, 03/13/2015 - 17:00
The Haiku community is developing a new operating system for personal computing. They were one of the twelve organizations who mentored students during Google Code-in 2014, our contest introducing 13 to 17 year old students to working in open source communities. Today, Scott McCreary reflects on the achievements of students who participated with Haiku.

GCI 2014.png
2014 was the fifth year for Google Code-in (GCI) and also Haiku’s fifth year participating. This was the first year where we offered beginner tasks designed to give students an easy way to get started. These tasks introduced students to booting and using Haiku, compiling Haiku, and using Haikuporter to build a package from a recipe file.
149 students completed at least one task with us this year and 36 students completed at least three tasks with us to earn a GCI t-shirt. Six students completed over 20 tasks each in a bid to become grand prize winners. Each mentoring organization selected two grand prize winners and three finalists. Our finalists were Augustin Cavalier, Markus Himmel and Chirayu Desai. (Chirayu was also a GCI 2013 winner with RTEMS.) Our winners were Josef Gajdusek and Puck Meerburg.
After getting started with beginner tasks, many students moved on to tasks which added improvements to Haiku or fixed bugs. Students contributed to HaikuWeather, SuperFreeCell, BookmarkConvert, and our CatKey editor. Many HaikuPorter recipes were added and updated, making it easier to bring existing software into Haiku. Several tickets were resolved, new GUI controls were created, new screensavers were added, and several apps were updated to make use of Layout Manager. In total, students completed a staggering 435 tasks for Haiku during the contest.
GCI grand prize winners are invited to San Francisco for an award trip. Last year, I had the pleasure of attending as a mentor along with our 2013 winners. This year, the winners selected Adrien Destugues (a former Google Summer of Code student, now one of our most active contributors) as the mentor they’d like to meet on the trip. I’m excited for the great time they’ll have!
I'd like to thank the 17 Haiku mentors and all 149 students who took part in GCI with Haiku this year. Also, a special thanks to our community members who were on IRC to help students during the contest. Finally, a shout-out to Stephanie Taylor and the rest of the team at Google for running Google Code-in and selecting Haiku to participate.
By Scott McCreary, Haiku GCI mentor
Categories: Open Source

Bidding farewell to Google Code

Thu, 03/12/2015 - 17:00

When we started the Google Code project hosting service in 2006, the world of project hosting was limited. We were worried about reliability and stagnation, so we took action by giving the open source community another option to choose from. Since then, we’ve seen a wide variety of better project hosting services such as GitHub and Bitbucket bloom. Many projects moved away from Google Code to those other systems. To meet developers where they are, we ourselves migrated nearly a thousand of our own open source projects from Google Code to GitHub.

As developers migrated away from Google Code, a growing share of the remaining projects were spam or abuse. Lately, the administrative load has consisted almost exclusively of abuse management. After profiling non-abusive activity on Google Code, it has become clear to us that the service simply isn’t needed anymore.

Beginning today, we have disabled new project creation on Google Code. We will be shutting down the service about 10 months from now on January 25th, 2016. Below, we provide links to migration tools designed to help you move your projects off of Google Code. We will also make ourselves available over the next three months to those projects that need help migrating from Google Code to other hosts.

  • March 12, 2015 - New project creation disabled.
  • August 24, 2015 - The site goes read-only. You can still checkout/view project source, issues, and wikis.
  • January 25, 2016 - The project hosting service is closed. You will be able to download a tarball of project source, issues, and wikis. These tarballs will be available throughout the rest of 2016.

Google will continue to provide Git and Gerrit hosting for certain projects like Android and Chrome. We will also continue maintaining our mirrors of projects like Eclipse, and others.

How To Migrate Your Data Off Google Code

The simplest way to migrate off of Google Code is to use the Google Code to GitHub exporter tool, which provides an automated way to migrate a project’s source, issues, and wikis to a new GitHub repo. Please note: GitHub’s importer will convert any Subversion or Mercurial Google Code projects to use Git in the process.

We also offer stand-alone tools for migrating to GitHub and Bitbucket, and SourceForge offers a Google Code project importer service.

If you encounter any problems using these tools, please log issues with us, contact, or email me directly ( We’ll also be closely tracking Hacker News, Reddit, and other popular forums to answer questions in public. We know this decision will cause some pain for those of you still using Google Code and we're sorry for that. We'll continue to do our best to make the migration process easy for you.

GitHub and Bitbucket are both looking forward to working with developers moving off of Google Code. They’ve been great to work with leading up to this announcement, so we’d like to thank those sites for their continued support of the community. There are some great options for people today that didn’t exist in 2006, and we look forward to helping you find the one that works for your project.

Chris DiBona, Director of Open Source

Categories: Open Source

CritiqueBrainz and rocking the Google Summer of Code

Fri, 03/06/2015 - 18:00
Student applications for Google Summer of Code 2015 will be opening on March 16th. University students interested in applying for this year’s program can get ready now by checking out the organizations taking part in this year’s program. Below, Robert Kaye shares the story of an outstanding student from last summer who made a big impact with the MetaBrainz Foundation, maintainers of the MusicBrainz service.

At MusicBrainz, we’re proud to collect and freely share information about all kinds of music. We strive to keep that information objective and factual, but being music fans, we can’t help but want to share our opinions sometimes, too. To do that, we developed a concept called CritiqueBrainz where we’d welcome people to write Creative Commons licensed, non-neutral point of view music reviews. Unfortunately, we didn’t have the resources to actually build it, nor was there any off-the-shelf software that would suit our needs.
When we took part in Google Summer of Code 2013, we put the CritiqueBrainz concept on our ideas page. It was too much to ask of one student to build a complete website with all the custom features we wanted in one summer. We accepted one student who wrote all of the features we requested as part of his successful GSoC project. However, over the summer it became clear that we needed to put a lot more work into the project before we could deploy the new site.
In 2014, we put CritiqueBrainz on our ideas page again and this time we asked for someone to finish the site in the first half of the summer, then to deploy, maintain and debug the site during the latter half of the summer. When the student application period opened, things got interesting. Several years ago, we added a requirement that applying students needed to submit a pull request to the project in order for us to consider their application. One student in particular submitted a pull request as part of his application. And then he submitted another. And another.
That one particular student submitted some truly excellent work as part of his application. So we accepted Roman Tsukanov, better known to us as “Gentlecat”, to finish the CritiqueBrainz project. At almost the instant we announced our accepted student proposals, his stream of pull requests started up again and never let up. As we were approaching the official start date of GSoC, the number of tasks left to finish the coding portion of the project was dwindling. Gentlecat even wrote a tool that parsed, cleaned up and imported 9,000 reviews that the BBC had given us. Things were looking excellent!
Given all that progress, we decided that the beginning of GSoC would be a perfect opportunity to release the first beta version of the CritiqueBrainz to the world. We gave Gentlecat access to one of our servers and let him deploy the code onto the server himself. With some guidance on best practices from us, he quickly got things up and running. Many people from the MusicBrainz community began writing reviews on the site, which naturally meant they found bugs and made tons of suggestions for improving the site.
Gentlecat plowed through the new tickets opened during the beta period and created pull request after pull request. My summer mornings consisted of reviewing his pull requests from the previous day. Gentlecat knocked out a long list of bugs while refactoring the code to have a better layout and improve readability. He added HTTPS support, found and fixed problems from the GSoC 2013 project, and even added integration with Spotify to support a hack I was creating at San Francisco Music Hack Day.
In the end, Gentlecat finished at least twice what he had promised to do during the span of GSoC. And the best part? He continues to pound out code and fixes, and he’s now a contributor to three of our projects.
He plans to apply for Google Summer of Code again this year, but he also feels that more people ought to be involved with CritiqueBrainz so he’s been contributing additional project ideas for other students to consider. If you’re a university student and CritiqueBrainz sounds interesting to you, please check out our GSoC 2015 ideas page.
Gentlecat rocked GSoC so hard that his nickname has become a term in our community: if you Gentlecat something, it means that you finished faster than anyone expected while going above and beyond what you promised to do. In other words: you rocked it. Hard!

by Robert Kaye, MusicBrainz GSoC mentor
Categories: Open Source

Mentoring Organizations for Google Summer of Code 2015

Mon, 03/02/2015 - 20:00
GoogleSummer_2015logo_horizontal.jpgWe are excited to announce the mentoring organizations that have been accepted for this year’s Google Summer of Code program. As always, we had many more great projects than we could accept. After reviewing 416 applications, we have chosen 137 open source projects, 37 of which are new to Google Summer of Code. You can visit our Google Summer of Code 2015 program website for a complete list of the accepted orgs.
Over the next two weeks, students interested in applying for the Google Summer of Code 2015 program can learn more about the 137 accepted open source projects. The student application period begins on Monday, March 16, 2015 at 19:00 UTC.
Interested? Start by reviewing the Ideas Page from each organization to learn about the project and how you might contribute. Some of the most successful proposals have been completely new ideas submitted by students, so if you don’t see a project on an Ideas Page that appeals to you, don’t be afraid to suggest a new idea to the organization! There are points of contact listed for each organization on their Ideas Page - students can contact the organization directly to discuss a new proposal. All organizations list their preferred method of communication on the organization homepage, available on the Google Summer of Code program website. We strongly encourage students to reach out to the organizations before they apply. Please see our Frequently Asked Questions page for more information.
Congratulations to all of our mentoring organizations! We look forward to working with all of you during this next Google Summer of Code!
By Carol Smith, Open Source Team
Categories: Open Source

Google Code-in 2014 wrap up with Wikimedia

Fri, 02/27/2015 - 18:00
The Wikimedia Foundation was one of the twelve mentoring organizations taking part in Google Code-in 2014, our contest introducing 13 to 17 year old students to working in open source communities. Below, Andre Klapper shares the accomplishments of several students who participated with Wikimedia during the contest.

The Wikimedia Foundation was proud to participate for a second time in Google Code-in. In this program, young students are introduced to free and open source software (FOSS) projects and invited to make practical contributions.
Between December 2014 and January 2015, 48 students successfully completed 226 Wikimedia tasks, supported by 30 mentors from our community. Those tasks include not only code development, but also documentation, research, and testing — leading to a wide range of achievements:

Thank you and congratulations to all the students who joined Wikimedia and supported its mission to freely share knowledge! Special kudos to Wikimedia’s two Grand Prize Winners: Danny Wu and Mateusz Maćkowski — and to our finalists Evan McIntire, Geoffrey Mon and Pranav Kumar! The full list of winners across all organizations can be found here.
We also wish to thank all our mentors for their generous commitment: we are especially grateful for the time they spent on weekends, coming up with task ideas, working with students and quickly reviewing their contributions. And last but not least, thank you to Google for organizing and running this contest, creating awareness of and interest in Free and Open Software projects.
We welcome more contributions to help improve our free and open software. Check out how you can contribute and our list of easy software bugs to start with.

By Andre Klapper, Wikimedia Foundation
Categories: Open Source

Introducing gRPC, a new open source HTTP/2 RPC Framework

Thu, 02/26/2015 - 20:25

(Cross-posted from the Google Developers Blog)

Today, we are open sourcing gRPC, a brand new framework for handling remote procedure calls. It’s BSD licensed, based on the recently finalized HTTP/2 standard, and enables easy creation of highly performant, scalable APIs and microservices in many popular programming languages and platforms. Internally at Google, we are starting to use gRPC to expose most of our public services through gRPC endpoints as part of our long term commitment to HTTP/2.

Over the years, Google has developed underlying systems and technologies to support the largest ecosystem of micro-services in the world; our servers make tens of billions of calls per second within our global datacenters. At this scale, nanoseconds matter. Efficiency, scalability and reliability are at the core of building Google’s APIs.

gRPC is based on many years of experience in building distributed systems. With the new framework, we want to bring to the developer community a modern, bandwidth and CPU efficient, low latency way to create massively distributed systems that span data centers, as well as power mobile apps, real-time communications, IoT devices and APIs.

Building on HTTP/2 standards brings many capabilities such as bidirectional streaming, flow control, header compression, multiplexing requests over a single TCP connection and more. These features save battery life and data usage on mobile while speeding up services and web applications running in the cloud.

Developers can write more responsive real-time applications, which scale more easily and make the web more efficient. Read more about the features and benefits in the FAQ.

Alongside gRPC, we are releasing a new version of Protocol Buffers, a high performance, open source binary serialization protocol that allows easy definition of services and automatic generation of client libraries. Proto 3 adds new features, is easier to use compared to previous versions, adds support for more languages and provides canonical mapping of Proto to JSON.

The project has support for C, C++, Java, Go, Node.js, Python, and Ruby. Libraries for Objective-C, PHP and C# are in development. To start contributing, please fork the Github repositories and start submitting pull requests. Also, be sure to check out the documentation, join us on the mailing list, visit the IRC #grpc channel on Freenode and tag StackOverflow questions with the “grpc” tag.

Google has been working closely with Square and other organizations on the gRPC project. We’re all excited for the potential of this technology to improve the web and look forward to further developing the project in the open with the help, direction and contributions of the community.

by Mugur Marculescu, Product Manager
Categories: Open Source

RE2/J: Linear-time regular-expression matching for Java

Tue, 02/24/2015 - 18:00
Today we’re announcing the public release of RE2/J: a pure-Java implementation of the popular RE2 regular expression library.
Although RE2/J is not always faster than java.util.regexp, its running time is always linear in the size of the input. Thus when matching larger inputs, especially against patterns containing a high degree of alternation, RE2/J may be dramatically faster. With a backtracking implementation such as java.util.regexp, it is not hard to construct a pathological pattern whose matcher would take years to run on some inputs, so RE2/J's performance guarantee makes it suitable for use in applications where the pattern is supplied by untrusted users, such as the clients of a web server.
If you are looking for a detailed technical discussion of the motivation for RE2 and RE2/J and the tradeoffs involved, please see “Regular Expression Matching Can Be Simple And Fast” and “Regular Expression Matching in the Wild”, both written by Russ Cox.
RE2/J is used widely by Java projects within Google. In many cases, it can be used as a drop-in replacement for java.util.regexp. We are pleased to be able to make this library available for public consumption.
Please head to RE2/J’s GitHub page to find out how to use it!
By James Ring, Google Engineering
Categories: Open Source

Google Summer of Code wrap up: Drupal

Fri, 02/20/2015 - 19:00
This week’s Google Summer of Code (GSoC) wrap up is from Matthew Lechleider at Drupal, a widely used content management system used for building websites.Hello from the Drupal community. We're an open source content management system used to create some of the coolest websites in the world and we're delighted to highlight just a few of our Google Summer of Code (GSoC) 2014 student projects. Not only did 12 students dedicate their summer contributing to Drupal, but most importantly they had fun. Many of the projects below may sound easy, but Drupal 8’s core architecture is a major upgrade from previous versions forcing many students to engineer code in our new systems.
Most students focused on porting frequently used modules to our upcoming version 8. Students such as Lucian Hangea from Romania worked on a vast array of functionality ranging from porting the Diff module with extensible new options to Jayesh Solanki from India who integrated Disqus comments. It is impressive to see contributions from students who are simply happy to be part of our open source community. We can only hope these talented young software geniuses stick around.
Shivanshu Agrawal from India not only created a working version of the Securesite module in Drupal 8 but also finished the Drupal 7 version port during his GSoC application phase. Thousands of users around the world wondered why Drupal 7 sites could not provide simple password authentication until a student made a simple change pushing the module from beta to stable.
Sachini Aparna Herath from Sri Lanka worked on adding support for mappings to Drupal 8. Helping websites and systems communicate in a standard way, enriches content allowing it to properly propagate the Internet. Because of Sachini, Drupal 8 site builders can seamlessly point/click markup to their sites using the RDF UI she built. Similar projects are available in previous versions of Drupal but not Drupal 8, making this project fairly difficult to accomplish in an early beta version.
Andrei-Marius Dincu from Romania ported XMLsitemap to Drupal 8. This module creates a sitemap which conforms to the specifications and helps search engines more intelligently crawl a website and keep their results up to date. Not only did Andrei port xmlsitemap.module to our newest beta versions of Drupal 8, he continues to provide contributions to previous versions of the module and even mentored students during Google Code-in.
Our community was invited to send two delegates to the GSoC Tenth Year Reunion event held in lieu of the annual mentor summit. We selected Angela Byron from Canada, one of our top GSoC Drupal alumni, and Chandan Singh from India, one of our best students from this summer. Angie's story of beginning with Drupal’s Quiz module as a GSoC student almost ten years ago and becoming one of the most important people in our community is now legendary. We're also excited about Chandan becoming a rockstar developer pushing Drupal to the next level as a promising new contributor. Beginning by leading development of the Entity Embed module as a GSoC 2014 student, Chandan continues to be actively involved in Drupal 8 core development, became one of our essential mentors in GCI 2014, and has returned to GSoC 2015 as a mentor.
A big thanks to all the students and mentors who helped make this summer a success and of course to the entire Drupal community for their amazing support. Last but certainly not least, thanks to Google for making it all possible. The entire open source community is forever in debt to the gift Google provides us with Summer of Code.
It's never too early to start planning for next year's Code-in or Summer of Code. Join our GSoC group on, our GCI group on, and chat with us via IRC in #drupal-google on Freenode to learn more.

by Matthew Lechleider, Drupal's GSoC Org Admin
Categories: Open Source

Google Code-in 2014: Magic in the Numbers

Thu, 02/19/2015 - 19:19
Google Code-in, our contest introducing 13-17 year olds to open source software development, wrapped up a few weeks ago with our largest contest to date: 658 students from 53 countries completed a record-breaking 3,236 tasks! We almost doubled the number of student participants in 2014 from the previous year’s contest (337). Working with 12 open source organizations, students wrote code, created and edited documentation, designed UI elements and logos, conducted research, developed screencasts and videos teaching others about the software, helped find and fix hundreds of bugs and many other interesting tasks.

General Student Stats
  • 49.1% of students completed 3 or more tasks (earning themselves a cool Google Code-in 2014 t-shirt)
  • 17.93% of students were female, up from 10.08% in 2013
  • This was the first Google Code-in for 552 students (83.9%)

CountriesIn the chart below we display the countries with the most student participants.
This year we had nine countries with students participating for the first time: Azerbaidjan, Brunei, Chile, Jamaica, Kazakhstan, Nigeria, Oman, Saudi Arabia, and Taiwan.SchoolsStudents from 397 schools competed in this year’s contest.

The five schools with the highest participation are:
  1. Dunman High School in Singapore for the second year in a row had the largest number of students completing tasks in the contest with 58, up from 20 last year!
  2. Govindram Seksaria Science P.U. College, Belgaum (GSS) in India had 49 students.
  3. Technical School Electronic Systems (associated with Technical University- Sofia) in Bulgaria was in the top five for the fifth straight year with 48 students.
  4. Sacred Heart Convent Sr. Secondary School, Jagadhri in India is in the top five again this year with 34 students.
  5. National College "Aurel Vlaicu" Orastie in Romania had 10 students.

Age of StudentsThe graph below shows the age breakdown of this year’s students.

GCI 2014 Student Age.png

Mentors and Organizations

  • 203 dedicated mentors from 40 countries (including six countries with mentors for the first time: Belize, Burma, Cameroon, Kazakhstan, Singapore and Uganda) helped guide students through the contest.
  • In true “pay it forward” style, 10 of this year’s mentors were former Google Code-in students. Great job!
  • The three organizations with the most tasks completed by students were FOSSASIA (587), Haiku (435), and BRL-CAD (383).

Every part of Google Code-in was bigger this year than ever before, but the most important part of the contest is that more students were introduced to open source software as teenagers. Congratulations students on all of your hard work! Thank you mentors and organization administrators for inspiring these young adults to contribute to open source software and welcoming them into your communities.

By Stephanie Taylor, Google Code-in Program Manager
Categories: Open Source

MapReduce for C: Run Native Code in Hadoop

Wed, 02/18/2015 - 19:00
MapReduce for C: Run Native Code in Hadoop

We are pleased to announce the release of MapReduce for C (MR4C), an open source framework that allows you to run native code in Hadoop.

MR4C was originally developed at Skybox Imaging to facilitate large scale satellite image processing and geospatial data science. We found the job tracking and cluster management capabilities of Hadoop well-suited for scalable data handling, but also wanted to leverage the powerful ecosystem of proven image processing libraries developed in C and C++. While many software companies that deal with large datasets have built proprietary systems to execute native code in MapReduce frameworks, MR4C represents a flexible solution in this space for use and development by the open source community.


MR4C is developed around a few simple concepts that facilitate moving your native code to Hadoop. Algorithms are stored in native shared objects that access data from the local filesystem or any uniform resource identifier (URI), while input/output datasets, runtime parameters, and any external libraries are configured using JavaScript Object Notation (JSON) files. Splitting mappers and allocating resources can be configured with Hadoop YARN based tools or at the cluster level for MRv1.  Workflows of multiple algorithms can be strung together using an automatically generated configuration. There are callbacks in place for logging and progress reporting which you can view using the Hadoop JobTracker interface. Your workflow can be built and tested on a local machine using exactly the same interface employed on the target cluster.

If this sounds interesting to you, get started with our documentation and source code at the MR4C GitHub page.  The goal of this project is to abstract the important details of the MapReduce framework and allow users to focus on developing valuable algorithms. Let us know how we're doing in our Google Group.
By Ty Kennedy-Bowdoin, Skybox team
Categories: Open Source

Google Summer of Code Wrap up: Catrobat

Fri, 02/13/2015 - 18:00
Today’s Google Summer of Code wrap up comes from Stefan Jaindl at the Catrobat Project, creators of the Catrobat visual programming language that makes mobile development accessible to kids.

The Catrobat Project has participated in the last four Google Summer of Code (GSoC) programmes since 2011. During these years, we have built a strong community with local and international students and received valuable contributions to our codebase. This year, we mentored seven students with great projects. We’d like to highlight a few of those projects.

Pocket Code is our Android app for creating and running programs written in Catrobat. Before GSoC 2014, it wasn’t possible to create a standalone APK from a Pocket Code program without some serious hacking. Roland Urbano created Gradle build automation tasks for Pocket Code, extended our Jenkins-CI platform to automatically build standalone APKs, and wrote appropriate tests. Users can now upload a program and request conversion into an APK, and we manually approve these requests and make the automatic conversion. Roland has continued working on the project after the summer and is adding more features.
The ScratchToCatrobat converter is one of our most important undertakings and exists to fill the gap between the well known Scratch system and our Pocket Code project. It empowers kids to run their self-made Scratch projects on their own phone. We had begun development of the converter prior to GSoC 2014, but thanks to Christian Wutte’s GSoC project we’re much closer to a "feature complete" stage. Christian further developed and adapted the converter to support many of our newer Pocket Code bricks (called blocks in Scratch) and formula operators (used in conditional blocks, e.g., if-bricks). The converter is still in alpha and not yet publicly available, but we look forward to sharing it in the future.
One common programming feature which wasn’t supported in the Catrobat language was support for lists. Bernhard Spitzer spent the summer building support into the Pocket Code IDE for creating lists and performing basic operations on them (e.g. add, insert, delete, replace list item). He also added list functions (e.g. number of list-elements, contain certain list-value, returning of a list item) to the Formula Editor. These are now available as bricks which users can add to their Catrobat programs and it brings us closer to feature parity with Scratch.

All seven of our students successfully completed the GSoC 2014 programme and delivered valuable results. We also want to give thanks for the great Google Summer of Code Reunion this year with very interesting unconference sessions, the trip to the Google Campus, the awesome dinner and other happenings such as the evening at the museum. It was definitely a great opportunity to get in touch with other Open Source projects.

By Stefan Jaindl, Catrobat Organization Admin
Categories: Open Source

Mentoring Organization Applications Now Being Accepted for Google Summer of Code 2015!

Mon, 02/09/2015 - 19:50

Do you represent a free or open source software organization looking for new contributors? Do you love the challenge and reward of mentoring new developers in your community? Apply to be a mentoring organization in the Google Summer of Code program! The organization application period is now open.

Now in its 11th year, Google Summer of Code is a program designed to pair university students from around the world with mentors at open source projects in such varied fields as operating systems, language translations, content management systems, games, and scientific software. Since 2005, over 8,500 students from more than 100 countries have completed the Google Summer of Code program with the support of over 480 mentoring organizations. Students gain exposure to real-world software development while earning a stipend for their work and an opportunity to explore areas related to their academic pursuits during their school break. In return, mentoring organizations have the opportunity to identify and attract new developers to their projects as these students often continue their work with the organizations after Google Summer of Code concludes.

The deadline for applying to be a mentoring organization for Google Summer of Code is Friday, February 20 at 19:00 UTC (11am PST). The list of accepted organizations will be posted on the Google Summer of Code site on Monday, March 2nd. Students will then have two weeks to reach out to the accepted organizations to discuss their project ideas before we begin accepting student applications on March 16th.

Please visit our Frequently Asked Questions page for more details on the program. For more information you can check out the Mentor Manual, timeline and join the discussion group. You can also check out the Melange Manual for more information on using the website. Good luck to all of our mentoring organization applicants!

By Carol Smith, Open Source Team
Categories: Open Source

Google Summer of Code Wrap up: Processing

Fri, 02/06/2015 - 18:00
Today’s Google Summer of Code (GSoC) wrap up comes from Daniel Shiffman at the Processing Foundation, the organization responsible for the Processing programming language used in many visual arts applications.

Processing is a programming language and development environment specifically tailored to the needs of visual thinkers and artists. Tens of thousands of people use Processing (often called “p5” for short) for design, performance, animation, cinema, and more. It’s also used in several computer science curriculums for its appeal to visually-oriented learners.
We participated in Google Summer of Code (GSoC) for the fourth time in 2014. The students working with us completed eleven projects, many of which are in active use by the Processing community while others will be part of the upcoming Processing 3.0 release. We’ll highlight just a few of these projects below, but we’re grateful to all the students who took part in GSoC with us.
PDE X for Processing 3.0 (Manindra Moharana)
PDE X is a Processing mode that introduces advanced IDE features like code completion, refactoring, live error checking, debugger and more. Manindra helped bring PDE X to a stable state, allowing it to become the default editor in Processing 3.0. Over 30 bugs were fixed as part of this effort. Manindra also added a few new features, including precise error highlighting using the Wagner-Fischer algorithm, manual control over code completion using Ctrl+Space, and a tab outline popup window.

p5.sound addon for p5.js (Jason Sigal)
Jason created the p5.sound addon for the p5.js library to bring the Processing approach to Web Audio. Its functionality includes audio input, playback, manipulation, effects, recording, analysis, and synthesis. Jason also wrote methods for file input / output and ported Processing's Table / TableRow classes to p5.js.

Contributions Manager for the PDE (Joel Moniz)
The Contributions Manager lets users easily install, remove, and update community-developed extensions from within the PDE (Processing Development Environment). Joel’s work this summer introduced new features to the Contributions Manager, such as the addition, removal and update of Tools and Modes without a restart, a new "examples-package"-type contribution, and highlighting contributions.

Sound for Processing 3.0 (Wilm Thoben)
Wilm began work on a lightweight sound library for Processing in late 2013 and improved on it during GSoC 2014 by adding new features, fixing bugs, and introducing cross-platform support. Sound is built on top of methcla, a C++ sound engine with native bindings for low latency support. Sound provides a collection of sound-synthesis objects, analyzers and effects.

by Daniel Shiffman, Processing Foundation
Categories: Open Source

Orgs Get Ready: Preparing for GSoC 2015 with Freifunk

Tue, 02/03/2015 - 18:00
We’re getting ready for Google Summer of Code 2015 and the first step is selecting the mentoring organizations that will take part. Organization applications will open next week on Feb 9th, so we encourage you to think about how your open source project could benefit from participation. Today, we welcome GSoC veteran Federico Capoano from Freifunk to share his enthusiasm with other orgs.

It may be the middle of winter in the northern hemisphere, but we at Freifunk are already fired up about the summer. Organization applications for Google Summer of Code (GSoC) 2015 will open soon, and we’ve been urging other open source projects to take part. We’ve participated in GSoC six times, but last year left us with a new level of passion and enthusiasm for the FOSS movement.
Freifunk is a decentralized organization which is building community networks in Germany. Community networks bring a FOSS-inspired approach to the computing networks we use every day: an open network built, owned and managed by private citizens who decide how they use it and which services to run on it. Our community has been participating in GSoC with other communities like Guifi (Catalunya and Spain), Ninux (Italy), and Wlan Slovenia (Slovenia). Together we are making great strides toward the realization of our dream networks.
In GSoC 2014, we had eight successful projects completed by students and mentors from many different parts of the world: Europe, South America and Asia. We don't think about it so often since our contributors are so busy and concentrated on their own projects, but I think it’s amazing that we had volunteers from all over the globe collaborating on open source software applied to open networks!
After the summer, I had the opportunity to attend the GSoC 2014 Tenth Year Reunion which was an incredible experience. I met my Freifunk friends and so many other talented and skilled people working in the FOSS field with many famous organizations from all over the world! Suddenly, I was overwhelmed with a feeling of excitement and enthusiasm!
We had a chance to meet the people of Google Open Source (Chris DiBona, Carol Smith, Cat Allman and Stephanie Taylor) and had the honour of meeting Linus Torvalds, Alfred Spector, Peter Norvig and Dirk Hohndel. Their talks at the event were incredibly inspiring and encouraging. The unconference sessions were very educational and we had an opportunity to discuss improving our GSoC experiences and even the program itself.
It’s difficult for me to describe the whole Reunion experience in words alone, but I can tell you something that might be valuable for you too: this experience ENFLAMED me. If there is something vital to the FOSS movement, it’s enthusiasm. Without enthusiasm and excitement, we wouldn't go anywhere. Coming home to Europe afterward, we brought all that excitement from our community back with us.
Since then, we’ve been conveying our experience to our friends and motivating people to start new exciting projects and prepare for 2015. I personally talked about GSoC and the #reunion14 at:
  • meetups and mailing lists
  • Roma JS (Javascript devs meetup in Rome)
  • PyRoma (Python devs meetup in Rome)
  • Cineca, an IT consortium for universities that also works with FOSS (the place where I work)
  • Fablab Roma Makers

We’ll also be attending several events this year if you’d like to meet up with us: FOSSASIA (Singapore, March 2015), Wireless Community Weekend (Germany, May 2015), OpenTech Summit (Germany, May 2015), Wireless Battle of the Mesh v8 (Slovenia, August 2015), and Chaos Communication Camp (Germany, August 2015).
Similarly, my German friends have talked and written a lot about our last GSoC experience in the different Freifunk meetups, mailing lists, and the main Freifunk blog.
I would like to use this opportunity to encourage everyone in the FOSS community to participate in Google Summer of Code in 2015 because we all need enthusiasm and excitement in our communities. What are you waiting for? Participate and make the best out of it, you won't regret it.
by Federico Capoano, Freifunk mentor
Categories: Open Source