Skip to content

Software Development News: .NET, Java, PHP, Ruby, Agile, Databases, SOA, JavaScript, Open Source

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

Google Open Source Blog
Syndicate content
News about Google's Open Source projects and programs.
Updated: 2 hours 45 min ago

Coming to America: how Google Summer of Code helped change my life

Wed, 02/03/2016 - 22:00
Today we feature a story about Weilin, a Google Summer of Code student turned PhD candidate. The 12th instance of Google Summer of Code is just around the corner! Visit our new program website at http://g.co/gsoc for more information about how you can get involved.
My name is Weilin Xu and I’d like to tell a personal story about my involvement with Google Summer of Code (GSoC) and just how the experience helped change my life.
I first learned of GSoC in 2011 when I was a CS undergraduate in Beijing. The program sounded great, and my first challenge was deciding which of the hundreds of participating open source organizations to apply to. I finally decided on the Nmap Security Scanner, a tool known for network discovery and security auditing. Nmap is even a movie star, but I chose it because of my fascination with networking and the new IPv6 protocol.
My biggest fear was that I might be unqualified to work on such a major project with millions of users. I didn’t have much experience in the open source community, and my English was really bad.
Meeting my GSoC mentor David Fifield (right) in San Francisco, May 2015
I applied anyway and I’m so glad that I did! The Nmap crew accepted my application and I was assigned a talented, friendly and patient mentor named David Fifield. David taught me how to use Git directly rather than just throwing me a tutorial to complete on my own. He believed that he could teach me in minutes online what could take me an hour to learn from an English article on my own. David also helped improve my English during our weekly online meetings and always encouraged me by pointing out how I’d improved. Working on Nmap with David that summer was terrific and gave me the confidence to succeed!
My project was improving Nmap’s IPv6 scanning features--particularly the host discovery system. The current IPv4 Internet’s address space is small enough to scan by brute force, but that is not possible with IPv6. So we researched and implemented other effective discovery methods, such as our targets-ipv6-multicast-slaac and targets-ipv6-multicast-echo scripts which discover link-local hosts within seconds using the IPv6 NDP protocol. Many of these techniques were already known to the networking/security community, but they were new to Nmap and that brought them into wider use. It was great to see the community appreciating these new features, and perhaps we even helped in spreading IPv6 adoption!
The Nmap GSoC experience was an important milestone in my life. It taught me critical development and research skills and it even helped me find a great job. Tsinghua University’s NISL lab offered me a full-time position which typically requires a master’s degree, but they made an exception because of my real-world GSoC experience!
Before GSoC 2011, continuing my graduate studies in the United States was never more than a dream. Study abroad is usually for wealthy Chinese families, not poor ones from rural areas. But David (a graduate student himself) encouraged me to apply and wrote a recommendation letter. I used the GSoC stipend to pay my graduate school application fees as well as testing fees for the GRE and TOEFL. After months of anticipation, I received great news— a full scholarship from the University of Virginia to research and study adversarial machine learning! My parents were very proud, and I moved to the U.S. for this exciting adventure.
I recently had the honor of meeting my mentor David in person, along with Nmap’s founder Fyodor at the “Nmap Secret Lair” in San Francisco. Fyodor took my picture with David that I’ve included in this post.
I would like to thank Google very much for organizing this fantastic GSoC program and my mentor David Fifield for being so supportive and patient and helpful. I’d also like to thank Fyodor for all of his help as well. Finally I’d like to thank my twin brother Guanglin Xu for introducing me to GSoC in the first place.
By Weilin Xu, PhD Candidate, University of Virginia
Categories: Open Source

Seesaw: scalable and robust load balancing

Sat, 01/30/2016 - 07:31
Like all good projects, this one started out because we had an itch to scratch…As Site Reliability Engineers who manage corporate infrastructure at Google, we deal with a large number of internally used services that need to be load balanced for scalability and reliability. In 2012, two different platforms were used to provide load balancing, both of which presented different sets of management and stability challenges. In order to alleviate these issues, our team set about looking for a replacement load balancing platform.
After evaluating a number of platforms, including existing open source projects, we were unable to find one that met all of our needs and decided to set about developing a robust and scalable load balancing platform. The requirements were not exactly complex - we needed the ability to handle traffic for unicast and anycast VIPs, perform load balancing with NAT and DSR (also known as DR), and perform adequate health checks against the backends. Above all we wanted a platform that allowed for ease of management, including automated deployment of configuration changes.
One of the two existing platforms was built upon Linux LVS, which provided the necessary load balancing at the network level. This was known to work successfully and we opted to retain this for the new platform. Several design decisions were made early on in the project — the first of these was to use the Go programming language, since it provided an incredibly powerful way to implement concurrency (goroutines and channels), along with easy interprocess communication (net/rpc). The second was to implement a modular multi-process architecture. The third was to simply abort and terminate a process if we ended up in an unknown state, which would ideally allow for failover and/or self-recovery.
After a period of concentrated development effort, we completed and successfully deployed Seesaw v2 as a replacement for both existing platforms. Overall it allowed us to increase service availability and reduce management overhead. We're pleased to be able to make this platform available to the rest of the world and hope that other enterprises are able to benefit from this project. You can find the code at https://github.com/google/seesaw.
By Joel Sing, Google Site Reliability Engineer
Categories: Open Source

Google Code-in 2015: our biggest year yet!

Tue, 01/26/2016 - 03:45

GCI 2015 logo.jpg

The 6th year of Google Code-in (GCI) came to a close today after an exciting seven week run. Currently, mentors from each of the 14 organizations are busy reviewing final work submitted by students. Each organization will pick two Grand Prize winners to receive a trip to Google’s California headquarters this June where they will meet Google engineers, see exciting product demos and enjoy a fun-filled day of adventure in San Francisco.

We would like to congratulate all of the new and returning students who participated in GCI this year. We are thrilled to have our biggest year yet: over the last seven weeks, 979* students from 65 countries completed 4755* tasks in the contest.

And finally, a HUGE thanks goes out to the heart of our program: the GCI mentors and organization administrators. These volunteers spend countless hours creating and reviewing hundreds of tasks and teaching teens about all facets of open source development (community standards, new and exciting technologies, code reviews, version control systems, IRC and everything in between). We could not run this program without you!
* These numbers will increase over the next couple of days as mentors review the final work submitted by students.

Mary Radomile, Open Source Programs
Categories: Open Source

J2ObjC 1.0 Release

Thu, 01/21/2016 - 22:30
We are pleased to announce the 1.0 release of J2ObjC, a Google-authored open-source compiler that lets iPhone/iPad applications use Java code. J2ObjC's goal is to support the sharing of an application's non-UI code (such as data access, or application logic) by writing it once in Java, then building it into the iOS application. This same code can be shared with the Android and web versions of the application (the latter using the GWT compiler), as well as with server-side code. J2ObjC is licensed under the Apache License, Version 2.0.J2ObjC is not a Java emulator, but instead translates Java to Objective-C classes that extend the iOS Foundation Framework. It supports the Java 8 language and runtime required by client-side application developers. JUnit and Mockito test translation and execution is also supported.  J2ObjC can be used with most build tools, including Xcode and Make, and there are Gradle and Maven plug-ins.J2ObjC does not translate user interfaces, as world-class apps need to have world-class user interfaces that adhere closely to the different iOS and Android design standards. J2ObjC instead focuses on writing common abstractions once, and verifying them with a common set of unit tests. This ensures that an app's features work the same across platforms, improving customer experiences. Teams developing multi-platform apps still need great engineers for each platform, but with J2ObjC they don't waste time rewriting each others' code.
Using continuous integration, J2ObjC helps product velocity. As each feature is added or bug fix made to the application's shared code, all platforms are automatically rebuilt and tested. And because common features are shared across platforms, a bug found on one platform is fixed once for all platforms.
Several of Google’s iOS applications use J2ObjC for these reasons, including Inbox by Gmail, Google Calendar, Google Docs, Google Sheets, Google Slides and Google My Business. Each team has dedicated iOS designers and engineers, but application logic common to all platforms is written once.
By Tom Ball, Google Engineering
Categories: Open Source

Ringing in the new year with Google Code-in

Fri, 01/01/2016 - 20:02
GCI 2015 logo.jpgToday marks the halfway point of the Google Code-in 2015 (GCI) contest. 690 teenagers from 63 countries have been busy working with 14 open source organizations completing an impressive 2,370 tasks. The number of students successfully completing tasks has already surpassed the total number for all of the 2014 contest!


Tasks that the students have completed include:
  • writing test suites
  • improving mobile UI
  • designing website landing pages
  • writing documentation and creating videos to help new users
  • working on internationalization efforts
  • fixing and finding bugs in the organizations’ software


Over 2700 students from 98 countries and 1800+ schools have registered for the contest. We look forward to seeing more great work from many of these students over the next few weeks. A big welcome to the students from four countries participating for the first time in GCI: Luxembourg, Moldavia, Mongolia and Uganda!  

The countries with the most students completing tasks so far are:
  • India - 194
  • United States - 170
  • Romania - 46
  • Singapore - 36
  • Canada - 21

There is still plenty of time to get involved with Google Code-in to earn digital certificates and a Google Code-in 2015 t-shirt. New tasks are being added daily to the contest site so if you don’t see something that interests you today, check back again every couple of days. Currently over 2,300 tasks are open for students to choose from.


The last day to register for the contest and claim a task is Sunday, January 24, 2016 with all work being due on Monday January 25, 2016 at 9:00 am PT.

Good luck to all of the students participating this year in Google Code-in!
By Stephanie Taylor, Google Code-in Program Manager
Categories: Open Source

Stay in touch with RememberMe

Fri, 12/18/2015 - 19:00
Occasionally on the open source blog we feature the personal projects of Googlers. Today we hear from Blair Kutzman, whose loss inspired the creation of RememberMe, an open source email reminder tool.
On June 2, 2014 my cousin Jeremy Monnett was killed in a plane crash. He left behind his two sons, Miles and Brooks, and wife Kate.  As much as we look to explain events like these, they unfortunately can happen at any moment and to anybody.Jeremy.jpgRememberMe is an open source project I created in memory of Jeremy that helps people keep in touch. The basic premise is simple — at a configurable interval (eg. daily, weekly, monthly, etc) an email arrives in your inbox reminding you to contact someone. The reply-to field on the mail is set so that replying will send your response to your loved one.

The loss of Jeremy reminded me of just how important it is to keep in touch with the people you love the most. I hope that by making this project public, others will create email address for young children, spouses, and journal their daily thoughts and daily activities. This provides a fabulous way both to chronicle their lives, and also share your daily thoughts.
Please check out the project on github and enjoy!
By Blair Kutzman, Google Engineering
Categories: Open Source

MusicBrainz’ Gelato Summit 2015

Tue, 12/15/2015 - 23:35
MusicBrainz is an open music encyclopedia maintained by a global community of users, collects metadata and makes it available to the public. The team just wrapped up their annual summit in Barcelona, sponsored in part by the Google Open Source Programs Office.
From October 30th through November 1st, the MusicBrainz Summit 15 took place in Barcelona, Spain. All of the MetaBrainz employees were there in addition to a handful of contributors from our community — two lead developers of BookBrainz, a long-time volunteer developer on MusicBrainz and the lead of AcousticBrainz. Between us, we represented seven countries and nine languages.Talking around the table.
We managed to cover a lot of ground on serious topics including how to avoid data/MBID loss, how to version data, dealing with record labels, unresolved style issues and integrating the various Brainz projects a bit better. The official notes for the summit are stored in a public Google Docs document. Feel free to read through and contribute your own comments.
Our group has discussed releasing the translations of MusicBrainz for years — but this time we actually did it! It is now available in English, German, Dutch and French. At some point in the not-too-distant future we will also enable the translation of all of our documentation.
We also used this time together to talk about how and why MBIDs get lost and what we can do to prevent this. As part of this discussion, we decided to make more edits “autoedits” for all users. The goal is to have shorter queue of open edits (although there has been a significant drop in open edits since mid November) but also to avoid losing MBIDs once they have been generated. More in depth discussion of our reasoning and some of the community’s response can be seen in the server release blog post and its comments.
Most of our time was productive, but you cannot visit a city like Barcelona and not enjoy the surroundings. In addition to the more obvious local sights, we were introduced to an amazing gelato place within walking distance from our summit. It’s definitely worth a visit! A MusicBrainz summit would not be complete without copious amounts of chocolate. A huge thank you to our attendees for supplying Ritter Sport direct from the factory!
Finally, a big thank you to Google and Spotify for helping to fund this meeting. It would have been a lot harder to bring all these people together from around the world without their (continued, no less!) support. Here’s to 2016 and summit 16.
By Frederik “Freso” Sandberg, MetaBrainz Community Manager
Categories: Open Source

Raising Sweet C++ Cane for Well-Dressed Animals

Thu, 12/10/2015 - 00:30
To celebrate the holiday season at Fun Propulsion Labs, we're trading our sushi mats and baking pans for candy canes and snowballs. Please join us for a special holiday-themed version of Pie Noon and Zooshi! Zooshi and Pie Noon are open source, cross-platform games built from a suite of libraries that eager C++ developers can use to build their own projects.

You can download and run Zooshi's Santa mode on Google Play and find the latest open source release on our GitHub page. The holiday version of Pie Noon is available on Google Play as Snowdown in Santa Tracker and on our GitHub page. Happy Holidays!

By Stewart Miles, Fun Propulsion Labs*

*Fun Propulsion Labs is a team within Google that's dedicated to advancing gaming on Android and other platforms.
Categories: Open Source

Google Code-in: may the source be with you

Mon, 12/07/2015 - 18:40
For the sixth year running, starting today, teens from around the world will have the chance to learn and develop new CS skills by competing in Google Code-in (GCI). By working on real software projects—with help from mentors along the way—students just starting out with Computer Science can begin to investigate and discover the countless opportunities at their fingertips through code.

During the seven weeks of Google Code-in, pre-university students (ages 13-17) can browse hundreds of tasks created by 14 open source organizations. Students then get to choose the tasks they find most intriguing. A wide variety of skills and interests are required to make any software project work, so the tasks in Google Code-in are designed to reflect that diversity. Students can choose to work on projects across documentation, coding, training, research, quality assurance, user interface and outreach tasks.

The 14 organizations students can work with during the contest encompass many fields: health care for developing countries, learning activities for elementary students, desktop and portable computing, the encouragement of young women in computer science, game development, to operating systems used in satellites and robots.
Each task has at least one mentor assigned to it - not only to review the student’s work, but to help answer questions along the way. Each organization also offers beginner tasks that give students who are newer to open source development an easy and clear place to get started. Another goal of the contest is to encourage students to find a coding community that they enjoy working with and hopefully become an active contributor for years to come.
Contest open screenshot.pngOver the last 5 years, over 2,200 students from 87 countries have successfully completed tasks by participating in Google Code-in. To celebrate CS Ed Week this year, please help us introduce even more young minds to open source software development through Google Code-in. To learn more about Google Code-in— including rules and FAQs—please visit the site and the Getting Started Guide.

By Stephanie Taylor, Open Source Programs
Categories: Open Source

Google Summer of Code wrap-up: LabLua

Fri, 12/04/2015 - 19:27
LabLua is a lab at PUC-Rio dedicated to research on programming languages, with emphasis on the Lua programming language. Lua is a powerful, fast, lightweight, embeddable scripting language that is used in many industrial applications and on several embedded systems and games.
This was our second year in Google Summer of Code (GSoC) and we feel our six completed projects made for a very successful outcome. Five of our projects were selected from our Ideas List, and one was proposed by the student herself! It was great to not only see students contributing code to the project, but to witness their enthusiasm for project design as well. Below are some brief highlights of this year's projects:
Port Lua Test Suite to NetBSD Kernel (Guilherme Salazar)The Lua interpreter was initially ported to the NetBSD kernel during GSoC 2010, allowing users to write kernel extensions using Lua scripts. During GSoC 2015, Guilherme worked on porting the Lua test suite to the NetBSD kernel. He adapted test scripts to eliminate dependencies on floating-point numbers and the operating system itself.
Guilherme also ported and reimplemented part of the Lua standard libraries (e.g., math and io) for satisfying test dependencies. Moreover, he found important bugs on the Lua kernel port.
Elasticsearch Lua client (elasticsearch-lua) (Dhaval Kapil)Elasticsearch is a distributed and scalable search engine written in Java that, besides its native transport protocol (Java to Java), offers a very complete REST API accessed through JSON. Dhaval built elasticsearch-lua, an initial client for the Lua language following a model similar to other existing clients, such as the clients written in Python and PHP.
Dhaval had the freedom to decide how to model the object oriented API and how to build tests during development. With elasticsearch-lua, programmers can now access most Elasticsearch functionalities, such as index, read, update, delete and search documents.
Port an SDL-based C++ open source game to CĂ©u (Alexander Tkachov)CĂ©u is a programming language that targets the development of reactive applications such as video games. It features first-class support for events and parallel compositions for lines of execution. For this project, we wanted to port an open source game of considerable size from C++ to CĂ©u.
Alexander chose the game Pingus (a Lemmings-like game for Linux) with a codebase of around 40k lines of code. An important requirement of our project was to do a "live port" of the game, in which the game remained fully functional while portions of it were ported to CĂ©u and the rest remained in C++. In this project, Alexander faced two main challenges: applying the CĂ©u idioms to the new code and keeping the old C++ code working across the language boundaries. Even though the code base was too big for a 3-month period, Alexander was able to port around half of the game, including its core (i.e., the game levels).
Add support for WSDL generation to LuaSOAP (Victor Dias)LuaSOAP is a library to ease the use of SOAP, the Simple Object Access Protocol. It provides a very simple API that converts Lua tables to and from XML documents. It offers two modules to help providing (server) and consuming (client) SOAP services.
Victor developed a module to help the generation of WSDL (Web Services Description Language) documents for the services implemented by the server. This new module traverses the service description (a Lua table, built by the programmer by hand, because Lua doesn't have this kind of information) and produces the WSDL description for it. This new module will be integrated into the next version of LuaSoap.
Design of TIER encoding and its implementation in Lua (Lukas Borin)TIER is an experimental language-independent binary encoding format, designed by this project's mentor, similar in purpose to formats like XML, JSON and Google's Protocol Buffers. TIER is designed to optionally support self-describing encodings that include meta-information about the serial structure of the encoded data; this meta-information can be used to support dynamic verifications or automatic decoding.
Lukas improved TIER's design and also wrote the first implementation of a Lua library that provides support for TIER encodings. He also wrote TIER's basic documentation and some tutorials on its usage. The TIER library supports custom mappings of the various TIER serial structures to specific Lua values. On top of this custom mapping support, Lukas also developed some predefined mappings for usual Lua values and a framework to facilitate the creation of mappings for usual Lua structures, such as sequences, maps, sets, etc.
Improvements to the Sailor Web Framework (Etiene Dalcol)Sailor is an MVC web framework for Lua. Its distinguishing feature is that it easily lets the programmer write Lua code for running both in the server and in the browser. It was created by Etiene Dalcol, and she submitted a project proposal to us with a series of planned improvements.
During GSoC 2015, she worked on three fronts for improving the framework: the first was a better test suite, the second was better documentation for the project, and the third was a series of improvements for the feature of running Lua code on the browser, with the possibility of having Lua code in the browser request new modules on demand from the server, and breaking the original dependency on also having node.js installed on the server.
All of the GSoC contributions are already committed to the Sailor project repository at Github, and the test suite is already part of Sailor 0.4, the latest public release of the project.
By Ana LĂşcia de Moura, Adjunct Professor at PUC-Rio and LabLua Administrator

Categories: Open Source

Google Summer of Code wrap-up: Drupal

Fri, 11/20/2015 - 19:00

Drupal is our featured Google Summer of Code organization this week. A long time Google Summer of Code and Google Code-in mentoring organization, they worked with 12 students last summer.drupal_logo-blue.pngHi, we're Drupal, a PHP-based content management system powering roughly 2% of all websites on the Internet and we participated in Google Summer of Code 2015 (GSoC) with a dozen projects! We have participated in 10 out of 11 years and this is no easy accomplishment. It is the dedication of each and every student who inspire us to volunteer our time year after year. Every GSoC excites us as we discover the next generation of programmers who will hopefully integrate themselves within our community.
Student work was focused on porting common modules to Drupal 8 (our newest version). Drupal 8 is a major change from a coding and systems architecture point of view, forcing students to resolve advanced logic issues on their own. Not only did students learn best practices of Drupal, but they adapted our new core technologies Symfony2 and Twig. Below is a list of our projects completed in GSoC 2015.
  • Shitiz Garg aka Dragoon: Hawk Authentication Integration for Drupal 8 - created a module to support HAWK, an HTTP authentication scheme using a message authentication code (MAC) algorithm to provide partial HTTP request cryptographic verification.

  • Lucian Hangea aka lhangea: Making Drupal smarter by learning from users’ behavior - provided a general framework to conduct experiments for advanced AI usages in Drupal by using a class of algorithms called multi-armed bandit algorithms which use reinforcement learning to display content variations based on user behavior.

  • Palash Vijay aka Palashvijay4O: Port Quick Tabs module to Drupal 8 - updated module to newest version providing an easy way to render tabs using Ajax as blocks of content.

  • Abhishek Kumar aka abhishek.kumar: Content Staging Solution for Drupal 8 - managed the transfer of content between sites based on CouchDB Replication Protocol in Drupal 8 with simple admin user interface plus command line options in Drush.

  • Sachini Herath aka sachini: Linked Data mapping tool for Drupal 8 and the Google Knowledge Graph - allows site builders to map their content to Freebase and WikiData from Drupal 8 with two options: 1. Map entity types such as nodes and taxonomy terms with Knowledge Graph. 2. Map content created using built-in WYSIWYG editor to Knowledge Graph.

  • Shivanshu Agrawal aka shivanshuag: Extending Site Audit and porting to Drupal 8 - updated an analysis platform that generates reports with actionable best practice recommendations in Drupal 8 plus implemented additional tests requested by top development agencies already familiar with the system who were surveyed by the student during proposal research.

  • Prateek Mehta aka prateekmehta: URL Embed Module for Drupal 8 - built a framework for CKEditor allowing users to display an embedded representation of a URL. Content of the URL can be video, images, rich text or a link.

  • Viktor Bán aka banviktor: Port Security Review to Drupal 8 - port new version of this module that helps site administrators automate testing for many of the easy-to-make mistakes that render your site insecure and create new tests to verify current best practices in security.

  • Alok Pandey aka zealfire: Port Print Module to Drupal 8 - update module to newest version utilizing APIs made available in Drupal 8 to make the architecture more stable and pre-processing of node's content into more robust printable formats.

Growth is critical to any community and we're proud that three of our 2014 students returned in 2015. In addition, student-alumni continue to become mentors after graduation and we're thrilled to see alumni-mentors returning annually who ultimately push this cycle of innovation forward. Most specifically, Drupal was lucky to find Chandan Singh aka cs_shadow who went from student to mentor and recently became our backup organization administrator. The system works!
A tip to mentoring organisations: utilize an org admin who does not mentor any specific project yet overlooks all projects with all students in a weekly check-in meeting. It was clear after the first and second weeks that our check-in meeting was effective. A few students needed a bit of guidance when it came to summarizing work and providing code to review (working with every student is a full time job in itself). However, by the end of summer students provided professional weekly reports that project managers in a real job would love to review. Plus students were able to share resources and peer review each other's work in meetings. Win win.
Thank you to Google for making all of this happen. It is exciting to watch this program grow and we're already planning for 2016. Learn more about contributing with us specifically at https://groups.drupal.org/google-summer-code or help us prepare for Google Code-in at https://groups.drupal.org/google-code-in.
By Matthew Lechleider, Drupal Org Admin
Categories: Open Source

Hungering for Game Utilities?

Wed, 11/18/2015 - 22:10
At Fun Propulsion Labs we spend some of our time building sample games to help demonstrate how to make easy-to-build, performant, cross-platform games. With the growth of Google Cardboard, we got to work and over many long evenings, feeding our animal hunger on sushi, we came up with Zooshi. Zooshi is an open source, cross-platform game written in C++ which supports:
  • Android, Android TV, Windows, OSX, and Linux
  • Google Cardboard
  • Google Play Games Services sign-in and leaderboards on Android
  • Level customization
Zooshi serves as a demonstration of how to build Android games using a suite of newly released and updated open source game technologies from Google:
  • Motive drives our Animation system, giving life and movement to the characters and environment.
  • CORGI, the Component Oriented Reusable Game Interface, is an Entity-Component system designed to allow users to define complicated game objects as collections of modular, custom-defined behaviors.
  • FlatUI is a straightforward immediate mode GUI system with a light footprint that makes building up user interfaces a breeze.
  • Scene Lab allows designers to design levels and edit entities from right in the game without needing to use an external editor.
  • Breadboard provides an easy to use node based scripting system for editing entity behaviors that's accessible to designers without deep knowledge of programming.
  • FPLBase is a cross-platform API layer, for abstracting low-level tasks like reading input and creation of graphical contexts.
As in our previous release, PieNoon, we also made extensive use of Flatbuffers, Mathfu, fplutil, and WebP.

You can download the game in the Play Store and the latest open source release from our GitHub page. We invite you to learn from the code to see how you can apply these libraries and utilities in your own Android games. Take advantage of our discussion list if you have any questions, and don’t forget to toss some sushi around while you’re at it!

Posted by Alex Ames, Fun Propulsion Labs*

* Fun Propulsion Labs is a team within Google that's dedicated to advancing gaming on Android and other platforms.
Categories: Open Source

New open source tools to import mail to Gmail

Tue, 11/17/2015 - 19:00
We have two new open-source projects to help people import their existing email into Gmail using the Gmail API: mail-importer and import-mailbox-to-gmail.

Do you have an old Thunderbird mail archive that you downloaded from an old email account? Do you want to move those old messages to Gmail so that you can use the Gmail app on your phone and still have access to everything? Then maybe mail-importer is for you! The project aims to make it easy for users to import mailbox files from their computer to Gmail. The project is in early development and only has command-line support right now.

Or maybe you are a Google Apps administrator trying to import all of your users’ old mbox files into Google Apps for Work? Then import-mailbox-to-gmail is probably what you’re looking for. It is a Python script that will do bulk imports of multiple mailboxes into multiple accounts in a domain. It's admin-managed, so your users won't have to individually import their archives. To migrate from other types of archives or mail servers to Google Apps, see the Help Center.

For developers, import-mailbox-to-gmail can be used as an example of how to perform domain-wide delegation for the Gmail API. mail-importer shows how to use batching and retries to efficiently query the Gmail API.

Interested? Want to contribute? Fork the projects on GitHub, send us pull requests, file issues and share your tricks on the wiki for each of the projects. See the CONTRIBUTING files for more details.
By Ian Flanigan, Software Engineer, Gmail and Liron Newman, Technical Solutions Engineer, Google for Work
Categories: Open Source

Introducing the Google Code-in 2015 Mentoring Organizations

Fri, 11/13/2015 - 22:00
Today we are happy to announce the 14 open source organizations that are participating as mentoring organizations for Google Code-in 2015! The contest, entering its sixth year, introduces 13-17 year old pre-university students to open source software development, giving them the opportunity to take the skills they have been learning in the classroom and apply them to a real open source software project. Experienced mentors are available to help the students if they have questions while working on each task.

Google Code-in officially starts for students on December 7, but before the contest begins, students can learn more about each of the mentoring organizations by clicking on the links below.
  • GCI vertical. 1142x994dp.pngApertium - rule-based machine translation platform
  • Copyleft Games Group - building game development platforms for tomorrow
  • Drupal -  content management platform
  • FOSSASIA - developing communities across all ages and borders to form a better future with Open Technologies and ICT
  • Haiku - operating system specifically targeting personal computing
  • KDE -  team that creates Free Software for desktop and portable computing
  • MetaBrainz - builds community maintained databases
  • OpenMRS - open source medical records system for the world
  • RTEMS - operating system used in satellites, particle accelerators, robots, racing motorcycles, medical devices and more
  • SCoRe - research lab that seeks sustainable solutions for problems faced by developing countries
  • Sugar Labs -  learning platform and activities for elementary education
  • Systers -  community for women involved in the technical aspects of computing
  • Ubuntu - open source operating system
  • Wikimedia Foundation - non-profit foundation dedicated to bringing free content to the world, operating Wikipedia


The mentoring organizations are currently busy creating hundreds of coding, documentation, user interface, quality assurance, outreach, research, and training tasks. The contest officially starts for students on Monday, December 7th at 9:00 PST.
For important contest information please check out the contest site for Contest Rules, Frequently Asked Questions and Important Dates. You can also find flyers and other helpful information including the new Getting Started Guide on the contest site. Join our announcement and discussion lists to talk with other students, mentors and organization administrators about the contest. For questions about eligibility or other general questions, you can contact us at gci-support@google.com.
By Stephanie Taylor, Open Source Programs
Categories: Open Source

Google Summer of Code wrap-up: RoboComp

Fri, 11/13/2015 - 20:33
We continue our Google Summer of Code wrap-up series with RoboComp, an open source robotics development project. Check out their students’ work below.

RoboComp is a robotics development framework that uses distributed components over the Ice communications middleware. RoboComp also provides a set of useful tools that facilitate the life-cycle of robotics software components.  

We participated in Google Summer of Code (GSoC) for the second time in 2015. The students working with us completed four projects and all showed great enthusiasm and dedication. One of the most remarkable experiences this year was watching the students work together, supporting one another in several situations. Their projects are briefly described below.

Symbolic Planning in perception: Mercedes PaolettiMercedes improved RoboComp’s symbolic planner and its AGM domain description language (pdf) to allow domains to include abstract actions. These actions can be used to reason about how robots can achieve tasks without specifying details until necessary (to some extent similar to lazy evaluation in general programming languages). For example, if you need to go to San Francisco you can plan two abstract actions: one to get to the airport and another to fly to SF. You then try to solve the first one, leaving the details of the second for later. In order to test it in a real robot with our cognitive robotics architecture CORTEX (pdf), Mercedes implemented a generalized inverse kinematics solver based on regularized non-linear optimization. This algorithm is used to move the arms of our robot Shelly (in the image).ursus3-hall-informatica.jpgRoboComp Building and Deployment System Design: Nithin MuraliNithin Murali made contributions in several areas. The current CMake building scripts were greatly improved, so that Debian packages can now be easily generated for the project. Additionally, Nithin created an Ubuntu PPA for the project with these packages. He also developed a workspace model and several tools to ease the deployment of components.

Computer Vision Components and Libraries Management: Kripasindhu Sarkar Kripasindhu wrote a library to manage computer vision algorithms in RoboComp. He added all the library support tools as well as most of the main 2D and 3D recognition algorithms that we use. Kripasindhu developed a polymorphic and repeatable class design, yielding code that is easier to use and understand.

RoboComp Tutorial, Social Management and Documentation: Rajath Kumar M.PRajath greatly improved RoboComp’s documentation system and social media presence. He had the difficult task of learning about RoboComp yet at the same time writing a long series of tutorials of increasing difficulty. He also fixed several bugs in the installation system and has enthusiastically verified that everything now works as intended. As a result, RoboComp is much easier to install and many more people across the world have started to use it.

by Luis Manso and Pablo Bustos, RoboLab, Universidad de Extremadura, Spain
Categories: Open Source

TensorFlow - Google’s latest machine learning system, open sourced for everyone

Wed, 11/11/2015 - 01:33
Cross posted from the Google Research blog 

Deep Learning has had a huge impact on computer science, making it possible to explore new frontiers of research and to develop amazingly useful products that millions of people use every day. Our internal deep learning infrastructure DistBelief, developed in 2011, has allowed Googlers to build ever larger neural networks and scale training to thousands of cores in our datacenters. We’ve used it to demonstrate that concepts like “cat” can be learned from unlabeled YouTube images, to improve speech recognition in the Google app by 25%, and to build image search in Google Photos. DistBelief also trained the Inception model that won Imagenet’s Large Scale Visual Recognition Challenge in 2014, and drove our experiments in automated image captioning as well as DeepDream.

While DistBelief was very successful, it had some limitations. It was narrowly targeted to neural networks, it was difficult to configure, and it was tightly coupled to Google’s internal infrastructure -- making it nearly impossible to share research code externally.

Today we’re proud to announce the open source release of TensorFlow -- our second-generation machine learning system, specifically designed to correct these shortcomings. TensorFlow is general, flexible, portable, easy-to-use, and completely open source. We added all this while improving upon DistBelief’s speed, scalability, and production readiness -- in fact, on some benchmarks, TensorFlow is twice as fast as DistBelief (see the whitepaper for details of TensorFlow’s programming model and implementation).
TensorFlow has extensive built-in support for deep learning, but is far more general than that -- any computation that you can express as a computational flow graph, you can compute with TensorFlow (see some examples). Any gradient-based machine learning algorithm will benefit from TensorFlow’s auto-differentiation and suite of first-rate optimizers. And it’s easy to express your new ideas in TensorFlow via the flexible Python interface.
Inspecting a model with TensorBoard, the visualization toolTensorFlow is great for research, but it’s ready for use in real products too. TensorFlow was built from the ground up to be fast, portable, and ready for production service. You can move your idea seamlessly from training on your desktop GPU to running on your mobile phone. And you can get started quickly with powerful machine learning tech by using our state-of-the-art example model architectures. For example, we plan to release our complete, top shelf ImageNet computer vision model on TensorFlow soon.

But the most important thing about TensorFlow is that it’s yours. We’ve open-sourced TensorFlow as a standalone library and associated tools, tutorials, and examples with the Apache 2.0 license so you’re free to use TensorFlow at your institution (no matter where you work).

Our deep learning researchers all use TensorFlow in their experiments. Our engineers use it to infuse Google Search with signals derived from deep neural networks, and to power the magic features of tomorrow. We’ll continue to use TensorFlow to serve machine learning in products, and our research team is committed to sharing TensorFlow implementations of our published ideas. We hope you’ll join us at www.tensorflow.org.



Posted by Jeff Dean, Senior Google Fellow, and Rajat Monga, Technical Lead 
Categories: Open Source

Google Summer of Code wrap-up: OpenKeychain

Fri, 11/06/2015 - 21:00
This week we feature OpenKeychain, an open source project that “helps users communicate more privately and securely”. Read more below about the exciting work their students completed last summer during Google Summer of Code.
OpenKeychain helps you communicate more privately and securely. It uses high-quality modern encryption to ensure that:
  • your messages can be read only by the people you send them to
  • others can send you messages that only you can read
  • these messages can be digitally signed so the people getting them are sure who sent them.
OpenKeychain is based on the well established OpenPGP standard making encryption compatible across all of your devices and operating systems.
This was OpenKeychain’s second year participating in the Google Summer of Code program. Two of our students did particularly great work which has been released in OpenKeychain 3.6.
Adithya Abraham Philip focused on OpenKeychain's interaction with keyservers. He implemented an automatic, periodic sync of keys that requires no user interaction. Since this potentially exposes a user's contacts, he implemented Tor and proxy support using the OkHttp library and the Orbot app. Adithya also implemented a more user friendly way to revoke and upload keys to keyservers. While adding the required settings for these features, he also created a UI redesign of the preference screen allowing better keyserver management.
Daniel Ramos worked on password alternatives by adding new key unlocking methods that the user can choose from. Specifically, this adds support for pattern, PIN, and NFC-based key protection mechanisms besides using a classic password. The methods have been implemented into OpenKeychain's key creation wizard. His work will be continued by themain developers and integrated in future versions of OpenKeychain.
By Vincent Breitmoser and Dominik SchĂĽrmann, main developers of OpenKeychain

Categories: Open Source

Google Summer of Code wrap-up: STE||AR Group

Fri, 10/30/2015 - 21:00
Today we are featuring the STE||AR Group, another Google Summer of Code veteran organization. Adrian Serio gives an overview of their four students summer projects below.
stellar.png
The STE||AR Group is an international team of researchers who aim to improve application scalability by more efficiently utilizing hardware resources available to developers. This summer has been an exciting time for the STE||AR Group’s Google Summer of Code (GSoC) mentors and students alike! We were very pleased with the dedication and effort of all five of our participants.
Our students made contributions to three of our software products:
  • HPX: a distributed C++ runtime system which comes with a standards-compliant API and allows users to scale their applications across thousands of machines
  • LibGeoDecomp: an auto-parallelizing library for petascale computer simulations which is able to take advantage of HPX to better adapt fluctuating workloads to the system
  • LibFlatArray: a highly efficient multidimensional array library which provides an object-oriented interface but stores data in a vectorization-friendly Struct-of-Arrays format.

Just like how these three products can work together as a tightly integrated stack, our goal with the GSoC projects was to create synergy between them and steer our development towards increasing the adaptivity and efficiency of our software. Below are the summaries of our student’s projects.
Implementation of a New Resource Manager in HPX: Nidhi MakhijaniThis project set out to properly assign hardware resources to executors: C++ objects that dictate the way a thread should be executed. Nidhi was able to allocate resources to an executor when it was created and return the resources when it stops. Additionally, Nidhi laid the groundwork for dynamic allocation where the resource manager can monitor and share resources amongst all of the running executors.
SIMD Wrapper for ARM NEON, Intel AVX512 & KNC in LibFlatArray: Larry XiaoVectorization is imperative for writing highly efficient numerical kernels. The goal of this project was to extend the already existing SIMD wrappers in LibFlatArray to more architectures (e.g. ARM NEON, Intel AVX512, etc.) and to extend the capabilities of these wrappers. Larry set out to study the different ISAs (Instruction Set Architectures), and make the library run efficiently on these architectures.
CSV Formatted Performance Counters for HPX: Devang BacharwarHPX provides users with a uniform interface to access arbitrary system information from anywhere in the system. Devang’s project allows users to request these counters in a CSV format. Additionally, he has enabled the ability to get timestamps with each value as well. These features will make it easier for HPX users to perform analysis on the performance data gathered from an application.
Integrate a C++AMP Kernel with HPX:  Marcin CopikThe HPX runtime system can coordinate the execution and synchronization of OpenCL kernels on arbitrary OpenCL devices, such as GPUs, in a system. In his GSoC project, Marcin used a C++ AMP compiler to produce an OpenCL kernel from a parallel algorithm implemented by HPX. Marcin integrated the Kalmar AMP compiler into the HPX build system, transformed a parallel for each algorithm into an OpenCL kernel, dispatched the kernel to a GPU and synchronized the result with a concurrently running HPX application.
A Flexible IO Infrastructure for LibGeoDecomp: Konstantin KronfeldnerIn LibGeoDecomp, users are able to read from and write to arbitrary regions of the simulation space. These operations are carried out by objects which we call Steerers and Writers. Over the summer, Konstantin added the ability for these Steerers and Writers to be dynamically created and destroyed. LibGeoDecomp is typically used on supercomputers, where jobs are executed non-interactively via a batch system. Konstantin's extensions enable users to interact with the application at runtime. They can view and modify the simulation model dynamically. The benefit of this is a significantly lower turnaround time for domain scientists who need to carry out many computational experiments.
By Adrian Serio, Scientific Program Coordinator, STE||AR Group
Categories: Open Source

Google Summer of Code wrap-up: Cesium

Wed, 10/28/2015 - 21:00
Today we are featuring Cesium, a three-time Google Summer of Code participant. Read more below about the fascinating work their students did with imagery this past summer. Cesium_Logo_Color.jpgCesium is a JavaScript library for creating 3D globes and 2D maps in a web browser without a plugin. It uses WebGL for hardware-accelerated graphics, and is cross-platform, cross-browser and tuned for dynamic-data visualization. Cesium first participated in Google Summer of Code (GSoC) in 2013. The bright students who have joined us through GSoC have made significant contributions to the Cesium community. This summer, our students worked on the following projects:NASA Worldview - Abhishek PotnisBringing Cesium to NASA imagery, Abhishek improved visualization of LIDAR profile data from NASA’s Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO). CALIPSO data is used to study atmospheric developments such as cloud formation and aerosol interactions, and can be extended to develop models for climate predictions among other possibilities. Using Cesium, Abhishek developed an interface that takes a user’s location input and displays the profile curtains nearest to that location. The user can select the time range for the data curtains, and the profile curtains near the location of interest will be refreshed accordingly. Perhaps most importantly, the interface shows the CALIPSO profiles in their natural “curtain” orientation in the context of traditional “flat” maps served by NASA’s Global Imagery Browse Services. This allows users to effectively combine the strengths of both types of maps for tasks such as determining the three-dimensional structures of clouds and dust storms. See the live demo and source code. GSoCBlog_Cesium.pngBy combining “flat maps” with “data curtains,” a user can visualize the height of clouds and dust storms above the Earth’s surface. In this case, cloud heights above western Africa are shown as red and yellow blobs in the generally blue LIDAR curtain. The CALIPSO satellite recorded the LIDAR curtain at approximately the same time as the Aqua satellite recorded the “flat map” below it.Cesium Support for GPX and Shapefiles - André NunesAndré joined us in 2013 as part of GSoC and worked on client-side support for KML, allowing users to easily visualize the many geographic data sets widely available in KML files. This year, he returned to tackle native Cesium support for GPS Exchange Format (GPX). GPX support will let anyone with a cell phone or other GPS device easily transfer their own outdoor activities (such as bike rides, running, boating, and even drone flights) into Cesium. Check out his GitHub pull request for the full technical details. Since he will be graduating from Técnico Lisboa in January, this is André’s final GSoC, but we hope he continues to contribute to Cesium as he embarks on his professional career!

Cesium Support for GML SFP - Ayush KhandelwalGeography Markup Language (GML) Simple Features Profile (SFP) is a common way of representing geospatial vector features such as points, lines, and polygons, plus accompanying metadata. In addition to being useful in its own right for spatial data visualization, GML SFP is commonly used to encode features retrieved from an OGC standard Web Feature Service (WFS) and to represent the result of a GetFeatureInfo call to a Web Map Service (WMS). This summer Ayush implemented support in Cesium for GML SFP.

By Sarah Chow, Cesium Organization Administrator
Categories: Open Source

Hungering for Game Utilities?

Wed, 10/21/2015 - 21:00
We here at Fun Propulsion Labs spend some of our time building sample games to help demonstrate how to make easy to build, performant, cross platform games. With the growth of Google Cardboard, and demand for useful resources, we got to work and over many long evenings, feeding our animal hunger on sushi, we came up with Zooshi.  Zooshi is an open source, cross-platform game written in C++ which supports:

  • Android, Windows, OSX, and Linux
  • Google Cardboard
  • Google Play Games Services sign-in and leaderboards on Android
  • Level customization

Zooshi serves as a demonstration of how to build Android games using a suite of open source game technologies from Google such as Motive, Breadboard, FlatUI, CORGI, Scene Lab, and FPLBase. As in our previous release, Pie Noon, we also made extensive use of Flatbuffers, Mathfu, fplutil, and WebP.
  • Motive drives our Animation system, giving life and movement to the characters and environment.
  • CORGI, the Component Oriented Reusable Game Interface, is an Entity-Component system designed to allow users to define complicated game objects as collections of modular, custom-defined behaviors.
  • FlatUI is a straightforward immediate mode GUI system with a light footprint that makes building up user interfaces a breeze.
  • Scene Lab allows designers to design levels and edit entities from right in the game without needing to use an external editor.
  • Breadboard provides an easy to use node based scripting system for editing entity behaviors that's accessible to designers without deep knowledge of programming.
  • FPLBase is a cross-platform API layer, for abstracting low-level tasks like reading input and creation of graphical contexts.

You can download the game in the Play Store and the latest open source release from our GitHub page. We invite you to learn from the code to see how you can implement these libraries and utilities in your own Android games. Take advantage of our discussion list if you have any questions, and don’t forget to toss some sushi around while you’re at it!
By Alex Ames, Fun Propulsion Labs*
* Fun Propulsion Labs is a team within Google that's dedicated to advancing gaming on Android and other platforms.
Categories: Open Source