Skip to content

Software Development News: .NET, Java, PHP, Ruby, Agile, Databases, SOA, JavaScript, Open Source

Methods & Tools

Subscribe to Methods & Tools
if you are not afraid to read more than one page to be a smarter software developer, software tester or project manager!

PHP

Laravel Dusk, Intuitive and Easy Browser Testing for All!

PHP Community Magazine - 7 hours 42 min ago

Sitepoint have interesting introduction to the latest Laravel 5.4 testing library : Dusk. Dusk is meant for end to end browser testing of modern JavaScript applications, and you will see in this article how you can test Ajax based applications. Overall, the article covered the configuration options necessary to get started, and the examples provided should help you fill in the gaps and give an overview of some of the other configuration options available.

With the release of Dusk, Laravel hopes to give its users a common API for browser testing. It ships with the default ChromeDriver, and if we need support for other browsers, we can use Selenium. It will still have this common testing API to cater to our needs.

Categories: PHP

Archive streaming library for PSR-7

PHP Community Magazine - Thu, 02/23/2017 - 09:14

ArchiveStream Message Body provides a memory efficient package for streaming Zip files as PSR-7 message. The package require PHP >=5.6.0, gmp extension, and psr/http-message. The only limitation is that only the Zip64 (version 4.5 of the Zip specification) format is supported and files cannot be resumed if a download fails before finishing.

To use it in Symfony HttpFoundation :

use Symfony\Component\HttpFoundation\StreamedResponse;

$stream = new Psr7Stream(new ZipReader($archive));

$response = new StreamedResponse(function () use ($stream) {
    while ($stream->eof() === false) {
        echo $stream->read($blockSize = 1048576);
    }
}, 200, [
    'Content-type' => 'application/zip',
    'Content-Disposition' => 'attachment; filename="file.zip"',
    'Content-Transfer-Encoding' => 'binary',
]);

You can use composer to add the package archive-stream to your dependencies :

composer require genkgo/archive-stream

More information at github, released under an MIT License.

Categories: PHP

Language Detection Library for PHP

PHP Community Magazine - Wed, 02/22/2017 - 22:04

In a linguistic diversity on the internet report published by UNESCO in 2009 it is noticed a steady year-on-year decline in the percentage of webpages in English from 75% in 1998 to 45% in 2005. W3techs.com indicates today that there are 171 languages used on the web, 52% for English, 133 are used by less than 0.1% of the websites. However the distribution of language spoken in the world is completely different since English speakers represent only 5% !

Finding new businesses, new customers, new leads, require necessarily speaking new languages, or at least understanding the language that others are using. That’s why a library for language detection is very important, and could open doors to bunch of new opportunities. A great implementation of a language detection algorithm in PHP is available today, and comes with text samples used for training and detecting text in 106 languages.

A basic usage of the library :

use LanguageDetection\Language;

$ld = new Language;

$ld->detect('Mag het een onsje meer zijn?')->close();

Result will be :

Array
(
    "nl" => 0.66193548387097,
    "af" => 0.51338709677419,
    "br" => 0.49634408602151,
    "nb" => 0.48849462365591,
    "nn" => 0.48741935483871,
    "fy" => 0.47822580645161,
    "dk" => 0.47172043010753,
    "sv" => 0.46408602150538,
    "bi" => 0.46021505376344,
    "de" => 0.45903225806452,
    [...]
)

The library is trainable which means you can change, remove and add your own language files to it. If your language not supported, feel free to add your own language files.

More information and download on the github repository, released under an MIT license.

Categories: PHP

Practice Design Patterns in PHP

PHP Community Magazine - Wed, 02/22/2017 - 09:02

DesignPatternsPHP is a collection of known design patterns and some sample code how to implement them in PHP. Every pattern has a small list of examples (most of them from Zend Framework, Symfony2 or Doctrine2). The problem with patterns is that often people do know them but don’t know when to apply which. So using these samples you should get more familiar with using the right pattern at the right time.

The patterns are structured in three different categories : Creational, Structural, and Behavioral. The repository include so far samples of 36 design patterns. A great collection written by Dominik Liebler and contributors, and released under an MIT license.

Documentation of the DesignPatternsPHP are available, and could be downloaded as PDF/ePub.

Categories: PHP

PHPStan, PHP Static Analysis Tool

PHP Community Magazine - Tue, 02/21/2017 - 10:36

PHPStan is a PHP static analysis tool that focuses on finding errors in your code without actually running it. It catches whole classes of bugs even before you write tests for the code. It moves PHP closer to compiled languages in the sense that the correctness of each line of the code can be checked before you run the actual line.

Supported features actually includes :

  • Existence of classes and interfaces in instanceof, catch, typehints, other language constructs and even annotations. PHP does not do this and just stays silent instead.
  • Existence of variables while respecting scopes of branches and loops.
  • Existence and visibility of called methods and functions.
  • Existence and visibility of accessed properties and constants.
  • Correct types assigned to properties.
  • Correct number and types of parameters passed to constructors, methods and functions.
  • Correct types returned from methods and functions.
  • Correct number of parameters passed to sprintf/printf calls based on format strings.
  • Useless casts like (string) 'foo'.
  • Unused constructor parameters – they can either be deleted or the author forgot to use them in the class code.
  • That only objects are passed to the clone keyword.

PHPStan requires PHP >= 7.0 and could be extended using Doctrine, Nette and Dibi. You can get it using composer :
composer require --dev phpstan/phpstan

For more information https://github.com/phpstan/phpstan

Categories: PHP

Introducing the Fistlab PHP Components

PHP Community Magazine - Thu, 02/16/2017 - 19:58

Fistlab is a new initiative to create a set of components in various programming languages, which however works almost the same way. This way it will be easier for developers to go around and play with another programming language without having to find a whole new set of components and dig into how it works.

The project started with PHP, and include already three components : Container, Database and Repository, in addition to few others planned. According to Mark Topper, author of the project, it may support PHP and Javascript for its first edition which is planned for August 2017 – and probably other languages later.

Current planned components include :

– Service Container
– Database Manager and Query Builder
– Router system
– Request and Response objects
– Filesystem
– Validator
– View System
– Controller System
– Model System

The initiative is not only new but very original, so it’s worth supporting. If you have any suggestion, or contribution, you can find more details in the contribution guide. Available under an MIT license.

Categories: PHP

Peachpie, The PHP Compiler and Runtime for .NET

PHP Community Magazine - Thu, 02/16/2017 - 13:10

Peachpie is a modern PHP compiler based on Roslyn by Microsoft and drawing from our popular Phalanger project. It allows PHP to be executed within the .NET framework, thereby opening the door for PHP developers into the world of .NET – and vice versa.

The project is a work in progress, so be aware that is not yet intended to run full applications. You can find more details on the project’s plan and progress on the roadmap page. The project’s goals :

  • Increased performance: Peachpie’s extensive type analysis and the influence of Microsoft Roslyn should provide an improved performance of PHP applications and components.
  • Security: since programs run within the standardized and manageable .NET or .NET Core environment, the code is fully verifiable without any unsafe constructs.
  • Cross-platform development: the project compiles legacy PHP code into portable class libraries, enabling developers to build cross-platform apps and libraries for Microsoft platforms.
  • Full .NET compatibility: compiled programs run on the reimplemented Peachpie runtime, fully compatibly with the PHP runtime.
  • Both-way interoperability: the project allows for hybrid applications, where parts are written in C# and others in PHP. The parts will be entirely compatible and can communicate seamlessly, all within the .NET framework.

You can follow the getting started guide, and get the code from github. Released under an Apache license.

Categories: PHP

Yahoo open-sources TensorFlowOnSpark, new distributed deep learning framework

PHP Community Magazine - Wed, 02/15/2017 - 10:32

Yahoo have just released TensorFlowOnSpark (TFoS), its latest framework for distributed deep learning on big-data clusters, to the open source community.

Nearly a year ago, Yahoo published CaffeOnSpark (CoS), an open source framework that allows distributed deep learning and big-data processing on identical Spark and Hadoop clusters. Yahoo uses CoS in many forms, and the framework has since been enhanced as a result of contributions from the community. However, it did not address the concerns of those using Google’s popular TensorFlow (TF), which was limited by its inability to be deployed on existing big-data clusters.

Yahoo developed TFoS to enable distributed TF execution on Spark and Hadoop clusters. Using a simple CLI and API, the new framework supports all types of TF programs, direct tensor communication among TF processes, scalable training of Inception image classification networks, and more.

Some initiatives have been noticed in the PHP community to expose tensorflow capabilities to PHP such as Tensile, however it’s still an experimental tool at its very early stages. PHP developers can simply implement a gRPC client for PHP that connect to tenserflow serving project to be able to talk to Tensorflow serving environment.

Below the press release :

Yahoo open-sources TensorFlowOnSpark, new distributed deep learning framework

February 15, 2017 – Yahoo has announced TensorFlowOnSpark, its latest open source framework for distributed deep learning on big-data clusters.

Deep learning (DL) has evolved significantly in recent years. At Yahoo, we’ve found that in order to gain insight from massive amounts of data, we need to deploy distributed deep learning. Existing DL frameworks often require us to set up separate clusters for deep learning, forcing us to create multiple programs for a machine learning pipeline (see Figure 1 below). Having separate clusters requires us to transfer large datasets between them, introducing unwanted system complexity and end-to-end learning latency.

Last year Yahoo addressed scaleout issues by developing and publishing CaffeOnSpark, our open source framework that allows distributed deep learning and big-data processing on identical Spark and Hadoop clusters. We use CaffeOnSpark at Yahoo to improve our NSFW image detection, to automatically identify eSports game highlights from live-streamed videos, and more. With the community’s valuable feedback and contributions, CaffeOnSpark has been upgraded with LSTM support, a new data layer, training and test interleaving, a Python API, and deployment on docker containers. This has been great for our Caffe users, but what about those who use the deep learning framework TensorFlow? We’re taking a page from our own playbook and doing for TensorFlow for what we did for Caffe.

After TensorFlow’s initial publication, Google released an enhanced TensorFlow with distributed deep learning capabilities in April 2016. In October 2016, TensorFlow introduced HDFS support. Outside of the Google cloud, however, users still needed a dedicated cluster for TensorFlow applications. TensorFlow programs could not be deployed on existing big-data clusters, thus increasing the cost and latency for those who wanted to take advantage of this technology at scale.

To address this limitation, several community projects wired TensorFlow onto Spark clusters. SparkNet added the ability to launch TensorFlow networks in Spark executors. DataBricks proposed TensorFrame to manipulate Apache Spark’s DataFrames with TensorFlow programs. While these approaches are a step in the right direction, after examining their code, we learned we would be unable to get the TensorFlow processes to communicate with each other directly, we would not be able to implement asynchronous distributed learning, and we would have to expend significant effort to migrate existing TensorFlow programs.

TensorFlowOnSpark

Yahoo’s new framework, TensorFlowOnSpark (TFoS), enables distributed TensorFlow execution on Spark and Hadoop clusters. As illustrated in Figure 2 above, TensorFlowOnSpark is designed to work along with SparkSQL, MLlib, and other Spark libraries in a single pipeline or program (e.g. Python notebook). TensorFlowOnSpark supports all types of TensorFlow programs, enabling both asynchronous and synchronous training and inferencing. It supports model parallelism and data parallelism, as well as TensorFlow tools such as TensorBoard on Spark clusters.

Any TensorFlow program can be easily modified to work with TensorFlowOnSpark. Typically, changing fewer than 10 lines of Python code are needed. Many developers at Yahoo who use TensorFlow have easily migrated TensorFlow programs for execution with TensorFlowOnSpark. TensorFlowOnSpark supports direct tensor communication among TensorFlow processes (workers and parameter servers). Process-to-process direct communication enables TensorFlowOnSpark programs to scale easily by adding machines. As illustrated in Figure 3, TensorFlowOnSpark doesn’t involve Spark drivers in tensor communication, and thus achieves similar scalability as stand-alone TensorFlow clusters.

TensorFlowOnSpark provides two different modes to ingest data for training and inference:

  1. TensorFlow QueueRunners: TensorFlowOnSpark leverages TensorFlow’sfile readers and QueueRunners to read data directly from HDFS files. Spark is not involved in accessing data.
  2. Spark Feeding: Spark RDD data is fed to each Spark executor, which subsequently feeds the data into the TensorFlow graph viafeed_dict.

Figure 4 illustrates how the synchronous distributed training of Inception image classification network scales in TFoS using QueueRunners with a simple setting: 1 GPU, 1 reader, and batch size 32 for each worker. Four TFoS jobs were launched to train 100,000 steps. When these jobs completed after 2+ days, the top-5 accuracy of these jobs were 0.730, 0.814, 0.854, and 0.879. Reaching top-5 accuracy of 0.730 takes 46 hours for a 1-worker job, 22.5 hours for a 2-worker job, 13 hours for a 4-worker job, and 7.5 hours for an 8-worker job. TFoS thus achieves near linear scalability for Inception model training. This is very encouraging, although TFoS scalability will vary for different models and hyperparameters.

RDMA for Distributed TensorFlow

In Yahoo’s Hadoop clusters, GPU nodes are connected by both Ethernet and Infiniband. Infiniband provides faster connectivity and supports direct access to other servers’ memories over RDMA. Current TensorFlow releases, however, only support distributed learning using gRPC over Ethernet. To speed up distributed learning, we have enhanced the TensorFlow C++ layer to enable RDMA over Infiniband.

In conjunction with our TFoS release, we are introducing a new protocol for TensorFlow servers in addition to the default “grpc” protocol. Any distributed TensorFlow program can leverage our enhancement via specifying protocol=“grpc_rdma” in tf.train.ServerDef() or tf.train.Server().

With this new protocol, a RDMA rendezvous manager is created to ensure tensors are written directly into the memory of remote servers. We minimize the tensor buffer creation: Tensor buffers are allocated once at the beginning, and then reused across all training steps of a TensorFlow job. From our early experimentation with large models like the VGG-19 network, our RDMA implementation has demonstrated a significant speedup on training time compared with the existing gRPC implementation.

Since RDMA support is a highly requested capability (see TensorFlow issue #2916), we decided to make our current implementation available as an alpha release to the TensorFlow community. In the coming weeks, we will polish our RDMA implementation further, and share detailed benchmark results.

Simple CLI and API

TFoS programs are launched by the standard Apache Spark command, spark-submit. As illustrated below, users can specify the number of Spark executors, the number of GPUs per executor, and the number of parameter servers in the CLI. A user can also state whether they want to use TensorBoard (–tensorboard) and/or RDMA (–rdma).

TFoS provides a high-level Python API (illustrated in our sample Python notebook):

spark-submit –master ${MASTER} \
${TFoS_HOME}/examples/slim/train_image_classifier.py \
–model_name inception_v3 \
–train_dir hdfs://default/slim_train \
–dataset_dir hdfs://default/data/imagenet \
–dataset_name imagenet \
–dataset_split_name train \
–cluster_size ${NUM_EXEC} \
–num_gpus ${NUM_GPU} \
–num_ps_tasks ${NUM_PS} \
–sync_replicas \
–replicas_to_aggregate ${NUM_WORKERS} \
–tensorboard \
–rdma

  • TFCluster.reserve() … construct a TensorFlow cluster from Spark executors
  • TFCluster.start() … launch Tensorflow program on the executors
  • TFCluster.train() or TFCluster.inference() … feed RDD data to TensorFlow processes
  • TFCluster.shutdown() … shutdown Tensorflow execution on executors

Open Source

Yahoo is happy to release TensorFlowOnSpark at github.com/yahoo/TensorFlowOnSpark and a RDMA enhancement of TensorFlow at github.com/yahoo/tensorflow/tree/yahoo. Multiple example programs (including mnist, cifar10, inception, and VGG) are provided to illustrate the simple conversion process of TensorFlow programs to TensorFlowOnSpark, and leverage RDMA. An Amazon Machine Image is also available for applying TensorFlowOnSpark on AWS EC2.

Going forward, we will advance TensorFlowOnSpark as we continue to do with CaffeOnSpark. We welcome the community’s continued feedback and contributions to CaffeOnSpark, and are interested in thoughts on ways TensorFlowOnSpark can be enhanced.
About Yahoo

Yahoo is a guide to digital information discovery, focused on informing, connecting, and entertaining users through its search, communications, and digital content products. By creating highly personalized experiences, Yahoo helps users discover the information that matters most to them around the world — on mobile or desktop. Yahoo drives value for advertisers by helping them engage with consumers online through the combination of data, content and technology. Yahoo is headquartered in Sunnyvale, California, and has offices located throughout the Americas, Asia Pacific (APAC) and the Europe, Middle East and Africa (EMEA) regions. For more information, visit the pressroom (pressroom.yahoo.net) or the Company’s blog (yahoo.tumblr.com).

Yahoo and Yahoo Messenger is/are the trademarks and/or registered trademarks of Yahoo! Inc.

All other names are trademarks and/or registered trademarks of their respective owners.

Categories: PHP

Cryptographically Secure PHP Development

PHP Community Magazine - Tue, 02/14/2017 - 08:15

Paragon Initiative Enterprises shared an interesting piece to Cryptographically Secure PHP Development. The article could be considered as additional resources to rules for programming cryptography code in C and point to some tips and techniques to :

– The Zeroth Rule of PHP Cryptography
– Easy Wins for PHP Cryptography Code
– PHP Cryptography: The Hard Parts

As conclusion, some cryptography best practices are simply not possible. To wit: PHP doesn’t allow you to perform direct memory management, so zeroing out memory buffers is not possible.

Furthermore, if a vulnerability is introduced somewhere else in the PHP interpreter (for example, via OpCache), there’s very little (if anything) you can do to mitigate it from a PHP script.

Read more at https://paragonie.com/blog/2017/02/cryptographically-secure-php-development

Categories: PHP

PHP 7.2: The First Programming Language to Add Modern Cryptography to its Standard Library

PHP Community Magazine - Tue, 02/14/2017 - 08:14

Officially, Libsodium will be part of the core extension of PHP 7.2 after RFC vote last week with 37 Yes and 0 No ! Sodium is a new, easy-to-use software library for encryption, decryption, signatures, password hashing and more.

Why it is modern cryptography ?

  1. Uses fast primitives designed to resist side-channel cryptanalysis (e.g. timing leaks, padding oracles).
  2. Exposes a high-level API that is simple and secure-by-default.

What does this mean to you ?

If you develop in PHP and can upgrade to 7.2 when it comes out, you get to enjoy modern cryptography as a part of the language itself. It will now be possible to design software that uses Ed25519 digital signatures (e.g. for automatic security updates) without requiring users to install an optional PHP extension.

More details in this article written by Scott Arciszewski.

Categories: PHP

Ways to use Redis database from PHP

PHP Community Magazine - Sun, 11/13/2016 - 18:10

On the Remi’s blog a small comparison of the various solutions to use a Redis database from PHP on Linux, all tested on Fedora 25 but should work on RHEL, CentOS or any other linux distribution. Remi tested redis extension, Predis library, phpiredis extension, and Predis library with phpiredis extension. The conclusion is pretty interesting :

I will be tempted to prefer the phpiredis extension when speed is the main priority, and the Predis library for code beauty.

Remi plan to help to the phpiredis extension author so a version can be published soon, preferably on PECL forge, to give it more visibility. By compiling results from different tests we can have something like :

Categories: PHP

Var_masterpiece the PHP var_dump Beautifier for Chrome

PHP Community Magazine - Sat, 11/12/2016 - 18:41

Var_masterpiece is a simple and useful PHP var_dump() beautifier that you can run on your Chrome browser. It simply turn the result of the var_dump function into a readable, parsable, foldable piece of art which makes PHP debugging much easier.

It simply convert something like :

var_dump

Into this :

vardump_beautifier

Add to Chrome var_masterpiece, the code source is also available on github so if you have any idea feel free to fork and contribute.

Categories: PHP

Apache : Why you should disable .htaccess

PHP Community Magazine - Sat, 11/12/2016 - 18:18

Sincerely, I can’t agree with James Hayden that the $500 millions Healthcare.gov website failed due to an Apache misconfiguration, because a half billion website should not exist at all ! But totally agree that security and performance wise, if there is not a real need for .htaccess it should be disabled. Usually .htaccess is needed on shared hosting only, otherwise it always better to use Apache main config files.

To configure your WordPress for example, simply copy paste your .htaccess content to your Apache config file then make sure you disable this feature from your server. There are more tips to tweak your Apache performance by disabling unused modules. Notice that these are not hidden tweak or something new, as it’s officially documented as the recommended best practices. So make sure you read very well the documentation, and of course there is no need to spend $500 millions on your next website !

Categories: PHP

turbo.js : Massive Parallel Computations in your Browser with GPGPU

PHP Community Magazine - Sat, 11/12/2016 - 18:18

turbo.js is a small library that makes it easier to perform complex calculations that can be done in parallel. The actual calculation performed (the kernel executed) uses the GPU for execution. This enables you to work on an array of values all at once. The library is compatible with all browsers (even IE when not using ES6 template strings) and most desktop and mobile GPUs.

Turbo.js is

  • Non-volatile : Data is always accessible from the GPU and JavaScript side. Typed arrays ensure data integrity and low transfer overhead. Each run debugs the kernel code on the fly.
  • Dynamic Data Layout: Work on a flat float array (example to the left) or use 4D vectors, utilizing four 32bit floats I/O per parallel operation. turbo.js takes care of data transformation for you.

The home page of the library provides a live benchmarking that show the difference in performance which is not huge but great enough. Performance is measured in vectors per milliseconds. The gauges display the number of vectors in thousands. So a result of 1.23 would indicate a speed of 1,230 vectors (4,920 floats) per ms. In this specific case, V/ms is equivalent to points/ms on the fractal surface.

687474703a2f2f692e696d6775722e636f6d2f42696951537a502e706e67

Turbo.js is designed for Desktop and Mobile, however since it’s using a feature that is not necessarily supported by all devices. Some phone manufacturers choose to use older GPU models for their devices that don’t support the required texture format turbo.js needs. That’s why, e.g., turbo.js works on Samsung S7 models using the Qualcomm Adreno 530, but not ones with the ARM Mali-T880 MP12.

More information and download at https://turbo.github.io/

Categories: PHP

Announcing SunshinePHP Conference #sunphp17

PHP Community Magazine - Fri, 11/11/2016 - 10:05

The SunshinePHP Developer Conference is coming to Miami, Florida on 2-4 February, 2017 hosted by the South Florida PHP community. The conference will host workshops for every level of PHP developer, the first day will include 8 PHP related tutorials and workshops that are each 3 hours of in-depth information. Next two days will have 5 keynotes and 40 PHP talks over 4 tracks.

SunPHP17 schedule is already here with lots of known names in the PHP community. Early bird registration is already open with special price for student. Save the date 2-4 February 2017 at the Embassy Suites Miami International.

More information at http://2017.sunshinephp.com/

Categories: PHP

PHP 7.0.13, 7.1.0 RC6 and 5.6.28 Released

PHP Community Magazine - Fri, 11/11/2016 - 09:02

The PHP development team announced today the immediate availability of PHP 5.6.28 and 7.0.13. two security releases with several bugs fixed and you are encouraged to upgrade.

Packages of PHP 7.0.13 are already available for Debian 8 “Jessie” on amd64 and i386 architectures, along with the following PECL extensions : APCu, APCu_bc, geoip, igbinary, imagick, memcached, mongodb, msgpack, redis, ssh2 and xdebug (Careful, PHP 7 support from some of them is still very young!).

Changes in 5.6.28 :

  • Core:
    • Fixed bug #73337 (try/catch not working with two exceptions inside a same operation).
  • Bz2:
    • Fixed bug #73356 (crash in bzcompress function).
  • GD:
    • Fixed bug #73213 (Integer overflow in imageline() with antialiasing).
    • Fixed bug #73272 (imagescale() is not affected by, but affects imagesetinterpolation()).
    • Fixed bug #73279 (Integer overflow in gdImageScaleBilinearPalette()).
    • Fixed bug #73280 (Stack Buffer Overflow in GD dynamicGetbuf).
    • Fixed bug #72482 (Illegal write/read access caused by gdImageAALine overflow).
    • Fixed bug #72696 (imagefilltoborder stackoverflow on truecolor images).
  • Imap:
    • Fixed bug #73418 (Integer Overflow in “_php_imap_mail” leads Heap Overflow).
  • SPL:
    • Fixed bug #73144 (Use-after-free in ArrayObject Deserialization).
  • SOAP:
    • Fixed bug #73037 (SoapServer reports Bad Request when gzipped).
  • SQLite3:
    • Fixed bug #73333 (2147483647 is fetched as string).
  • Standard:
    • Fixed bug #73203 (passing additional_parameters causes mail to fail).
    • Fixed bug #73188 (use after free in userspace streams).
    • Fixed bug #73192 (parse_url return wrong hostname).
  • Wddx:
    • Fixed bug #73331 (NULL Pointer Dereference in WDDX Packet Deserialization with PDORow).

And below changes in 7.0.13 :

  • Core:
    • Fixed bug #73350 (Exception::__toString() cause circular references).
    • Fixed bug #73181 (parse_str() without a second argument leads to crash).
    • Fixed bug #66773 (Autoload with Opcache allows importing conflicting class name to namespace).
    • Fixed bug #66862 ((Sub-)Namespaces unexpected behaviour).
    • Fix pthreads detection when cross-compiling.
    • Fixed bug #73337 (try/catch not working with two exceptions inside a same operation).
    • Fixed bug #73338 (Exception thrown from error handler causes valgrind warnings (and crashes)).
    • Fixed bug #73329 ((Float)”Nano” == NAN).
  • GD:
    • Fixed bug #73213 (Integer overflow in imageline() with antialiasing).
    • Fixed bug #73272 (imagescale() is not affected by, but affects imagesetinterpolation()).
    • Fixed bug #73279 (Integer overflow in gdImageScaleBilinearPalette()).
    • Fixed bug #73280 (Stack Buffer Overflow in GD dynamicGetbuf).
    • Fixed bug #72482 (Ilegal write/read access caused by gdImageAALine overflow).
    • Fixed bug #72696 (imagefilltoborder stackoverflow on truecolor images).
  • IMAP:
    • Fixed bug #73418 (Integer Overflow in “_php_imap_mail” leads to crash).
  • OCI8:
    • Fixed bug #71148 (Bind reference overwritten on PHP 7).
  • phpdbg:
    • Properly allow for stdin input from a file.
    • Add -s command line option / stdin command for reading script from stdin.
    • Ignore non-executable opcodes in line mode of phpdbg_end_oplog().
    • Fixed bug #70776 (Simple SIGINT does not have any effect with -rr).
    • Fixed bug #71234 (INI files are loaded even invoked as -n –version).
  • Session:
    • Fixed bug #73273 (session_unset() empties values from all variables in which is $_session stored).
  • SOAP:
    • Fixed bug #73037 (SoapServer reports Bad Request when gzipped).
    • Fixed bug #73237 (Nested object in “any” element overwrites other fields).
    • Fixed bug #69137 (Peer verification fails when using a proxy with SoapClient)
  • SQLite3:
    • Fixed bug #73333 (2147483647 is fetched as string).
  • Standard:
    • Fixed bug #73203 (passing additional_parameters causes mail to fail).
    • Fixed bug #71241 (array_replace_recursive sometimes mutates its parameters).
    • Fixed bug #73192 (parse_url return wrong hostname).
  • Wddx:
    • Fixed bug #73331 (NULL Pointer Dereference in WDDX Packet Deserialization with PDORow).

It’s also notable that these two releases are marked by the latest release candidate RC6 for PHP 7.1.0 that you are encouraged to download and test. PHP 5.6 which support end by December 31th 2018, is entering soon to its end of life and will get only security updates after January 2017. While PHP 7.0 will keep getting active support until December 2017.

php-supported-versions

There is no fixed date now for the availability of the 7.1.0, however there should be something around 2? November according to the release timetable.

Categories: PHP

How PHP7 Boosted Tumblr Performance

PHP Community Magazine - Thu, 11/10/2016 - 21:29

On the Tumblr engineering blog a post about PHP 7 at Tumblr, and how the migration from PHP5 to PHP7 boosted the performance of Tumblr. The project started as a Hackday in 2015, and at that time not all extensions supported PHP7 but it still works ! Then the real work started this spring as things goes more stable, the migration finished in August where all production servers was updated.

An interesting tool that helped in the migration is Phan, it helped finding code in the codebase that would be incompatible with PHP 7. Of course, in addition to a suite of unit and integration tests.

The CPU load on the servers decrease at least 50%

The performance was noticed immediately after the migration, the latency was dropped by half and the CPU load on the servers decrease at least 50%, often more.

tumblr_inline_og2suygoio1qzyw0r_1280

tumblr_inline_og2sv6unmf1qzyw0r_1280

Language features

PHP 7 also brings a lot of fun new features that can make the life of the developers at Tumblr a bit easier. Some highlights are:

  • Scalar type hints: PHP has historically been fairly poor for type safety, PHP 7 introduces scalar type hints which ensures values passed around conform to specific types (string, bool, int, float, etc).
  • Return type declarations: Now, with PHP 7, functions can have explicit return types that the language will enforce. This reduces the need for some boilerplate code and manually checking the return values from functions.
  • Anonymous classes: Much like anonymous functions (closures), anonymous classes are constructed at runtime and can simulate a class, conforming to interfaces and even extending other classes. These are great for utility objects like logging classes and useful in unit tests.
  • Various security & performance enhancements across the board.
Summary

PHP 7 is pretty rad! and congratulation to Tumblr team and to the PHP community !

Categories: PHP

Guide to Automatic Security Updates For PHP Developers

PHP Community Magazine - Thu, 11/10/2016 - 09:10

Most of the software security vulnerabilities known to man are preventable by careful development practices. For example, SQL injection can be prevented by separating the user-provided data from the SQL query. We’ve covered quite a few of these topics before. Read A Gentle Introduction to Application Security if you’re not familiar with web application security.

However, even if you’re trying to do everything right, eventually we all make mistakes and ship exploitable software.

The Case for Automatic Updates

Once a security bug exists in your customer’s networks, preventing a security breach involves a lot of moving parts, but most importantly:

  1. Identifying the security bugs before criminals do.
  2. Fixing the security bugs you’ve identified.
  3. Getting the patch deployed in all of your customer’s networks.

Consider the timeline below:

The Life and Death of a Software Vulnerability (Timeline)

The time between point 0 and point 1 might be years. For the Linux kernel, it’s about 5 years on average. There are a lot of ways to reduce this number, and most of them involve automated testing and manual code review from security experts.

The time between point 1 and point 2 might be weeks or months. Our average vulnerability identified-to-fixed time window is less than 24 hours. Most organizations do not enjoy the same agility or expertise.

Most development teams have no control over the time between points 2 and 3 — the amount of time it take for the fix to be applied (and the vulnerability to be neutralized) after the update is available.

By making updates manual rather than automatic, you’re forcing your customers to take all the responsibility for making sure that your mistakes don’t hurt their business. Only a very small minority of your customers might prefer the responsibility of verifying and applying each update themselves.

By and large, most people do not have the security awareness, time management, and discipline to undertake this responsibility 24/7/365.

Automatic security updates reduce the interval between points 2 and 3 from possibly infinite to nearly zero. That’s clearly a meaningful improvement over manual patch management.

What’s the Practical Risk in Outdated Software?

The problem of outdated software is well-studied by the information security industry. According to Verizon’s 2015 Data Breach Investigations Report (PDF), for example, when a software vulnerability was used…

99.9% of the exploited vulnerabilities were compromised more than a year after the CVE was published.

Earlier this year, Wombat Security reflected on similar studies in an article titled Out-of-Date Software and Plug-ins Compound End-User Risk. An article on Help Net Security examines findings from F-Secure and echoes the problem of companies relying on out-dated software and putting their users at risk.

The danger of outdated software is supported by both the data and by simple logic: If criminals are aware of a specific vulnerability in a software product, it doesn’t matter that the vendor published a security patch if most of the companies that use their product will remain vulnerable when criminals want to exploit it.

It Seems So Obvious, But…

If 99.9% of real-world software exploits are preventable by keeping software up-to-date, and the danger of unpatched vulnerabilities is well-known among security professionals, the obvious question is, “Why doesn’t everything update itself automatically?”

Implementing a secure automatic update mechanism is a nontrivial engineering feat that most programmers don’t even know where to begin addressing. And often the ones that think they do are uninformed about the risks and complexity.

Elements of a Secure Automatic Update System

Automatic updates are a specific instance of a more general problem: secure code delivery. But in addition to the problems involved with securely delivering code from the developer to the user (with a healthy dose of distrust towards the delivery infrastructure), you have additional engineering challenges to contend with due to the automatic nature of the whole process.

The components of a secure automatic update system are as follows:

Offline Cryptographic Signatures

Cryptographic signatures provide a (very strong inductive) proof of authenticity.

Every update package should be digitally signed, using (listed in order of preference):

  1. RFC 7479 (Ed25519, or Ed448 — modern elliptic curve cryptography).
  2. Ed25519 in conjunction with SPHINCS-256 (post-quantum cryptography).
  3. RFC 6979 (Deterministic ECDSA).
  4. GnuPG, which shines in this use-case and doesn’t require you to even care which algorithm it’s using.
  5. RSASSA-PSS with 2048-bit keys, SHA256, and MGF1-SHA256 and e = 65537.
  6. Foot-bullety ECDSA (i.e. you’re responsible for generating random nonces and dodging invalid curve attacks, and worrying about whether or not your curve is twist-secure). This is ranked lower than RSA because most libraries that only implement non-deterministic ECDSA are riddled with side-channel attacks. But the most glaring problem is the risk of nonce reuse. The standard was broken. RFC 6979 fixes it. RFC 7479 uses safer building blocks.

For PHP developers: you can get Ed25519 from libsodium or secure RSA from EasyRSA. The PEAR bindings for Crypt_GPG are available via Composer:

composer require pear/crypt_gpg

Since cryptographic signatures require a key pair (one secret/private key and its corresponding public key), simply publish/hard-code your public key with your software and use it to verify the signatures of update files.

Reproducible Builds

Reproducible builds is a concept that’s mostly relevant in compiled languages, where you can compile the source code on your own machine and get a byte-for-byte identical binary blob as the one the developer published and signed. This makes it more difficult for an attacker (or the developer) to slip in a trojan without being detected.

Reproducible builds necessarily require software projects to be open source. Closed-source (proprietary) software cannot satisfy this requirement.

For PHP developers, if your deliverable is a PHP Archive (.phar file), your users can use Pharaoh to verify that the .phar you publish and the .phar built on their machine have no differences. If any differences are found, you can then examine them to verify that they’re benign. (This is more useful than just comparing checksums.)

Decentralized Authenticity / Userbase Consistency Verification

To prevent targeted attacks:

  1. All updates should be checked into an append-only audit log.
  2. The local copy of the audit log should be validated among trusted peers.

The audit log can either mean implementing something like Certificate Transparency or simply publishing a SHA-384 hash of the file’s contents and additional metadata onto a cryptocurrency ledger.

When downloading an update file, after checking the cryptographic signature (which is a strong assertion that it was signed by the developer), one should check with their trusted peers to validate that they see the same copy of the audit log. If they unanimously respond affirmatively, then you can rule out any likelihood of a silent, targeted attack being deployed (especially if the infrastructure doesn’t know who your trusted peers are; Tor helps here).

This feature only guarantees that attacks can’t be targeted to one user without alerting the entire network to the presence of the attack. (It does not, however, stop an attacker who breached the infrastructure and pilfered the signing keys from releasing an update in the first place.)

In addition to being forensically useful, it’s a deterrent for the most sophisticated adversaries (whom, for the most part, want their activities to remain undetected).

Transport Layer Security

For every developer that believes the solution to curl http://foo | sh is simply curl https://foo | sh, there must be three that believe a GnuPG signature obviates the need for HTTPS. This is incorrect. You need both.

Consider the following scenario:

  1. You release version 5.5.1 of your software widget, with a valid signature.
  2. A vulnerability is found in 5.5.1, so you patch it and release version 5.5.2, with a valid signature.
  3. Because you’re using HTTP, an active attacker replays version 5.5.1 and its valid signature when an update request is emitted.
  4. Your users’ system remains on the vulnerable version until the attacker lets them upgrade.

This is known as a replay attack, and cryptographic signatures alone don’t remedy them.

The knee-jerk response to a replay attack is to upload your secret key to the server and do online signing (either with timestamps or challenge-response authentication). This defeats the purpose of an offline digital signature: To be secure even when the server hosting the files is breached.

You could also build a really complex system of offline signatures (see: most Linux distros and their RELEASES.gpg or equivalent). But why bother adding more complexity to your life when you can simply use HTTPS instead of HTTP and man-in-the-middle-assisted replay attacks disappear.

Mirrors and Other Availability Concerns

The other way for a criminal to prevent a security release from getting applied automatically to targets of interest would be to simply launch a sustained Distributed Denial of Service attack against the update server.

Fortunately, if you’ve implemented all the features above, your protocol already has enough resilience to allow you to decentralize your infrastructure. This means you could either:

  • Have a wide range of mirrors, hosted in a diverse range of networks with a diverse portfolio of internet service providers, OR
  • Simply upload copies of every update file to the BitTorrent network and let your users seed updates.

Trojan horse malware wouldn’t a concern for your users either way; your software is already verifying cryptographic signatures with your public key.

Either strategy greatly increases the cost of a sustained DDoS attack and reduces the likelihood of it having the intended effect.

The mail critical security updates must go through.

Separation of Privileges

Once you’ve obtained an authentic, peer-verified, reproducible-from-source copy of the security update from a highly-available decentralized networking infrastructure… what to do with it?

There are two schools of thought here:

  1. The update process can be initiated by the HTTP process when a user accesses the application that needs updating.
    • This requires the software to be able to write to itself, which greatly reduces your defense-in-depth to traditional web vulnerabilities, but ensures out-of-the-box automatic security updates are available for users who don’t have the knowledge or time to set up privilege separation.
  2. The update process must be initiated by a different user than apache/www-data (or equivalent). The software is unable to write to itself, and in fact can only be overwritten by another user account.
  • This is more secure, but carries the risk of more people disabling automatic security updates out-of-the-box.

Your threat model should dictate which avenue you pursue. CMS Airship went with the first option because we believe the risk of a 1day vulnerability to be much higher than “the source code is, effectively, world-writable” concerns.

If you have a more sophisticated ops team, by all means go with the second approach. Just keep in mind AviD’s Rule of Usability:

Security at the expense of usability, comes at the expense of security.

An intermediate solution does exist here: Your HTTP process writes to a named pipe (or a world-writable file), and another process (e.g. a daemon or a cron job) kicks off the update process as a more privileged user when this pipe/file is written to.

Secure Automatic Update Implementations

Although not immediately useful for PHP developers, The Update Framework is definitely the best starting point for secure automatic updates. In addition to the features covered in the TUF Specification, it supports a process for augmenting the protocol.

If you’re using CMS Airship, read up on how we implemented secure automatic updates.

Summary

Automatic software updates are a boon for security, not a detriment, so long as the following precautions are taken:

  1. Updates must be cryptographically signed.
  2. Updates must be reproducible from the source code.
  3. Update consistency must be decentrally verifiable.
  4. All communications must be conducted via TLS (i.e. HTTPS not HTTP) to prevent MitM-DoS attacks.
  5. Update files must be mirrored wide and far to prevent DDoS attacks.
  6. Updates may be handled by a separate user account than the one used by the HTTP server for better defense-in-depth.

Original Article by the Paragon Initiative Enterprises

Paragon Initiative Enterprises is an Orlando-based company that provides software consulting, application development, code auditing, and security engineering services. We specialize in PHP Security and applied cryptography.

Published under a creative commons license : CC-BY-SA 4.0 Intl
Categories: PHP

eBook : What you need to know about Angular 2

PHP Community Magazine - Thu, 11/10/2016 - 08:37

angular2As part of the FREE eBook every day by Packt, you can explore today an eBook written by Olivier Manickum titled “What you need to know about Angular 2”. Hurry offer expires in 16 hours as I am writing this post.

Well, the eBook is only 27 pages giving you just the jist to get started with Angular 2 :

  • Cover the fundamentals and the things you really need to know, rather than niche or specialized areas : installation, step by step guide to create an application
  • Assume that you come from a fairly technical background and so understand what the technology is and what it broadly does : It explains clearly components, directives and templates.
  • Focus on what things are and how they work : How to use data bindings, services, and metadata.
  • Include 3-5 practical examples to get you up, running, and productive quickly. In addition to a chapter dedicated to dependency injection.

AngularJS was created by Google in 2010 to address the need of creating awesome single-page applications. Angular2 embraces ES6 and TypeScript to create beautiful JavaScript code that gets compiled for the browser.

Claim your free eBook now : What you need to know about Angular 2. You can also subscribe to get daily updates of free eBooks.

Disclaimer : This is not a sponsored post. It’s true that I was previously judge in the PacktPub open source awards – Javascript category, however I am sharing this with you because I think it could be useful to some readers. HaPHPy reading !

Categories: PHP

Papernic, Open Source Document Management & Collaboration System

PHP Community Magazine - Wed, 11/09/2016 - 21:51

Pappapernic-logoernic is an open source document management collaboration system that will transform the way your business manages documents and information. It provides support to multi-user environment, documents upload, with classification, categories, expiry date, notes etc… The application is written in Symfony, so even if it doesn’t fit exactly with your need it could be an excellent platform to build on it. Released under a GPL v3 license.

papernic

Online Demo available : https://panel.papernic.com/en/ in Turkish also.

Categories: PHP