Frank Karlitschek_ http://karlitschek.de own clouds, free software and privacy Sat, 29 Aug 2015 13:00:52 +0000 en-US hourly 1 http://wordpress.org/?v=4.3 User Data Manifesto 2.0 launched http://karlitschek.de/2015/08/user-data-manifesto-2-0-launched/ http://karlitschek.de/2015/08/user-data-manifesto-2-0-launched/#comments Sat, 29 Aug 2015 13:00:52 +0000 http://karlitschek.de/?p=533 In October 2012 I announced the first version of the User Data Manifesto during the Latinoware Keynote in Brazil. The idea was to define some basic right that all users should have in the digital age. This was still before the Snowden revelations. But it was already very clear that the privacy and security is at risk by cloud services and SaaS solutions that totally ignore the rights and interests of their users. So the idea was to try to define what this rights should be in the internet age.

The version 1.0 was instantly very popular and I got a ton of positive feedback and support. But over the time it also became clear that a few things could be expressed in a simpler and clearer way. So the idea came up to do a revision of the manifesto based on all the feedback.

During last years ownCloud Contributor Conference Hugo Roy from FSFE and ToS;DR, Jan-C. Borchardt and I started to work on the version 2. Now one year later I’m super happy to say that Hugo launched the new version of the manifesto during the ownCloud Contributor Conference keynote here in Berlin just a few minutes ago.
This is the result of a lot of discussion and the input from a lot of people and organizations. I’m also super proud to say that several well known organization are official launch partners of this 2.0 version of the manifesto and support the manifesto and the ideas behind this. This supporters are:

  • KDE
  • GNOME
  • Free Software Foundation Europe
  • Netzpolitik.org
  • X-Lab
  • ToS;DR
  • Spreed
  • ownCloud
  • More information about the manifesto can be found here

    I hope that this Manifesto helps to promote the importance of privacy, data protection, security and control over the own data in the cloud age.
    If your organization, company or open source project wants to help to push this forward and want to support this manifesto then please send me a message and we will add you to the list of supporters.

    ]]>
    http://karlitschek.de/2015/08/user-data-manifesto-2-0-launched/feed/ 2
    Announcing the draft Federated Cloud Sharing API http://karlitschek.de/2015/08/announcing-the-draft-federated-cloud-sharing-api/ http://karlitschek.de/2015/08/announcing-the-draft-federated-cloud-sharing-api/#comments Thu, 06 Aug 2015 13:04:32 +0000 http://karlitschek.de/?p=526 Here you go: link

    I believe that federation of cloud services is the next important step for truly secure and flexible file sync and share cloud software. Because of that we are working for a while on the needed technologies and now have the first draft of an open specification ready.
    The goal of the Federated Cloud Sharing API is to be a common language for sharing files across different file sharing server implementations. That only works if a wide audience provides input, which is why we started the Open Cloud Mesh initiative and have already been working with and talking about earlier drafts to partners and other open source projects.

    The document describes an API which consists of two parts, WebDAV for file transfer and a simple REST based API to initiate sharing and exchange metadata. It is a very simple and pragmatic model, re-using as much existing technology as possible to ease implementation and migration. Rather than re-thinking the entire infrastructure of the web to enable federation, it combines the existing model of email (username @ server) and file transport protocols.

    For more information and background on the architecture, read my earlier blog post.

    For more general information on Federation, see my preceding post about the Next Generation File Sync and Share technology

    This draft is more than theory as it is already implemented in ownCloud, being introduced as ‘server to server sharing’ with ownCloud 7 and matured to its current state which you can try out in ownCloud 8.1. See the video below for Federated Cloud Sharing in action.

    ]]>
    http://karlitschek.de/2015/08/announcing-the-draft-federated-cloud-sharing-api/feed/ 1
    A huge step forward by doing 1000 small steps http://karlitschek.de/2015/07/a-huge-step-forward-by-doing-1000-small-steps/ http://karlitschek.de/2015/07/a-huge-step-forward-by-doing-1000-small-steps/#comments Tue, 07 Jul 2015 12:22:41 +0000 http://karlitschek.de/?p=519 Today the ownCloud community released ownCloud 8.1. You might think it sounds like a smaller release because we called this 8.1 instead of 9.0 but you’d be wrong. The contributor community hasn’t slowed down, quite the opposite. By the end of last year we decided to do smaller releases instead of huge ones. Because of that we will jump to a major version number only once a year and will increase the minor version number for the other releases. So this one is called 8.1 but it’s a major new version. So what is in 8.1?

    I try to talk to ownCloud users as much as possible. At all the trade shows and conferences that I visit during the year, on github by working with the incoming bug report and feature requests and by talking with the big ownCloud customers and prospects. There is one request that I hear more than others: “please concentrate on the quality and the performance of ownCloud at the next release”. So this is exactly what we did for ownCloud 8.1 which we announce today.

    The big new hallmark feature of ownCloud 8.1 is that *there is no hallmark feature*. We improved a lot of smaller things and annoyance and streamlined ownCloud in a lot of areas. But there is not one big thing that is standing out. Yet we improved a ton of small things that, at the end, result in a huge step forward for the experience of our users.

    Let me go over some of the changes in detail:

    • Quality. We closed over 500 bugs during the ownCloud 8.1 development phase. This is a huge number, bigger than ever before. A lot of small annoyances and problems in certain scenarios or architectures are gone.
    • Performance. We improved the WebDAV and sync performance significantly. We did some performance tests and we see over 4x the performance for many common file operations and syncing compared to 8.0, which was already a lot faster than 7.0. Additionally the Desktop syncing client got a lot of performance improvements in the last few releases. One example is the increased number of parallel requests it does while communicating by the servers. This all leads to a significantly improved sync performance.
    • Reliability in super high load situations. If you run ownCloud instance with ten thousands of users accessing much of that data in parallel on an ownCloud cluster, we discovered you could see concurrency problems when the system is under heavy load. This is very unlikely to happen in smaller instances but needs to be adressed of course. To improve this we implemented a transactional filesystem locking system that is available in 8.1 as preview. The plan is to have this on by default for 8.2.
    • The encryption app got rewritten. The code is now way more reliable, stable, easier to maintain and extend then before. This is especially important for some enterprise users who want to implement their own encryption system or integrate their enterprise key server. With the new modular encryption system this is now possible.
    • The LDAP configuration wizard is a lot more user friendly and more responsive when configuring very big directories with huge numbers of users and groups.
    • We deprecated the support for PHP 5.3 which is no longer supported by the PHP community for a while. Additionally we deprecated the support of the ownCloud server running on Windows. Windows as a client is still fully supported of course. This eliminates a lot of bugs that we had to deal with because of these legacy systems.
    • The App Store system is now improved and easier to use. Users can install apps easier than before, including the many experimental 3rd party apps for testing. Please note that experimental and not reviewed apps should not be used on a production instance because of potential security and data loss risks.
    • We extended the Updater app to support update channels. You can choose if you want to get daily updates, beta releases stable releases or if you want to be even more conservative and update a few days after the stable releases. The details are described in this blog post
    • ownCloud Documents got updates to the latest WebODF version which makes everything more stable and usable. More information can be found in a blog post coming later this week.
    • We did a bunch of smaller UI improvements to make ownCloud easier to use. So there are way more links from the UI to the corresponding documentation pages, more configuration hints, better debug messages and overall design improvements.
    • Additionally we improved a lot of code in the background that leads to a higher unit test coverage and overall better quality.
    • We also started to implement automatic smashbox into our development workflow. This is of course additionally to all the other automatic CI and unittests that we already do. This helps a lot to make sure that syncing is always stable in al circumstances. Smashbox was developed by CERN and is now pushed forward be the ownCloud community and CERN together. Please see this blog post and this blog post for details.

    More information about this awesome release an be found here.
    I think that this is the most polished, stable, easiest to use and faster ownCloud release so far. And I have to say that I’m super proud about the awesome work that the fast growing ownCloud contributor community did here.

    More and more people get interested in ownCloud as users, contributors or developers of 3rd party apps. This makes me very happy.

    If you want to experience this community and start to contribute then you should join us in August in Berlin for the ownCloud Contributor Conference. More information can be found here

     

    ]]>
    http://karlitschek.de/2015/07/a-huge-step-forward-by-doing-1000-small-steps/feed/ 5
    Building a platform http://karlitschek.de/2015/06/building-a-platform/ http://karlitschek.de/2015/06/building-a-platform/#comments Sat, 13 Jun 2015 01:07:15 +0000 http://karlitschek.de/?p=516 From the very beginning ownCloud has had bigger ambitions then just being a file sync and share tool. This is apparent from the name ownCloud. Today, we have more than our documents and photos online. Our social networks and shared thoughts, our appointments and shopping lists, audio and video conversations all happen and are stored somehwere ‘in the cloud’, all connected. You can comment on a song you like for others to see or share an appointment with co workers. ownCloud means to give you a chance to bring all that back under your control!

    But even with the scalability of the open source development model, ownCloud could not cover the almost infinite number of things happening online. Rather than creating one huge application which does it all, ownCloud was designed to have a relatively small core and implement most of its functionality in independent apps. Examples include very central functionality like sharing, versioning, the trash bin, encryption and LDAP support.

    Such a modular architecture makes complex software like ownCloud more manageable and easier to develop and maintain. But we also introduced an interface for admins to manage the apps of an ownCloud instance.

    We did that because ownCloud was meant to be a platform for third party applications. Sync and share is absolutely the core feature of ownCloud but we always wanted to build something more. A platform with an ecosystem where other developers can build applications on top.

    Examples for apps are:
    – New ways to authenticate users like LDAP
    – New connectors to storage backends like FTP or Dropbox
    – New file handling features like versioning or encryption
    – Features like Calendar, Contacts, Mail, Music and so on.
    – Completely new features like RSS readers, Document editing, chat or password management.

    Growing the app ecosystem

    At the moment a default ownCloud server installation already includes a handful of apps that provide basic functionality. Additionally around 50 third party apps are available to install to get more functionality. This is really great. But let´s think bigger here.

    If you look at other platforms like Windows, Mac, iOS, Android or Drupal, KDE and GNOME then you see that there are hundreds to hundreds of thousands of independent apps available to run on top. Saying that we could reach the same number in the short term would be unrealistic. But we definitely want to have a lot more than we have today and become a real platform.

    What do we have to do in ownCloud to grow from the current 50 to let´s say 1000 apps in the near future? For that we need to follow three important principles in developing ownCloud related to mindset, process and structure: supporting decentralization, building a great technology platform and having clear user communication and promotion.

    Supporting Decentralization

    If we are to build an ecosystem, we have to accept that we will not have everybody who works on ownCloud in one place. Not every app will be on github, not every app developer will use our testing tools or coding guidelines, will be on our mailing list or can ask questions in person. Key elements here are our infrastructure and communication.

    Infrastructure

    Not every app will be developed on Github or even the ownCloud Github organization. Our github repo is open for everyone who wants to developer their app close to the core code and team to benefit from help and code reviews from other people. Anybody can mail our developer list to request a new repository. But they don’t have to – apps can be developed anywhere and we have to adjust our processes to this.

    For example, even if we’d update all applications in the ownCloud github organization to an API change, we can’t deprecate the API! We don’t know and can’t control who else depends on this interface and if we do not take this bigger view, we won’t reach the thousands of third party apps we want.

    This also means we have to deal with third party apps not using our translation system, unit tests, coding guidelines or anything else part of our infrastructure! Our API’s and the way we design ownCloud has to fit with any way of working.

    Communication

    The second element to supporting a decentralized ownCloud ecosystem is our communication. We will have to realize that the core team can’t know every third party developer in person. It is very useful to be able to talk to the core developer of an app to discuss an API or UI change or a possible bug. This open communication is one of the core benefits of an open development process. But it is pretty clear that this doesn’t scale if you have > 1000 developers and apps. So every attempt to centralize the communication or base it on personal contact or friendship alone will fail or limit the growth of ownCloud. So every communication of APIs, best coding practices, release schedules or any other information that effect app developers has to happen in a way that works for a big ecosystem.

    This means there is a challenge with getting announcements to third party developers. Not all of them will be on our mailing-lists or IRC channels or will attend our meetings or conferences. You might say that it is enough to discuss topics that effect app developers on this channels or events. But we have to realize that not everyone can be a core member of the ownCloud contributor community to follow this channels. All relevant information and announcements has to be made available to all app developers via our documentation, blog posts or other scalable ways of communication.

    Building a Great Technology Platform

    The first step to making ownCloud a platform is technology. We need to offer a well documented, stable and complete base for third party developers to build on.

    Stable APIs

    It is key here to realize that not everybody has as much time to spend on ownCloud as core developers do. Updating an application because one of the APIs it uses has been changed is not only a chore without bringing any user benefits, it can take a long time or never happen if an app is not actively maintained.

    Platforms like Windows, Mac, Android and so on provide a stable API that is backward compatible for years. We have to do the same if we want to be a useful platform and grow a big ecosystem. iOS and Android apps from 5 years ago still work with the latest release. Windows and Mac software from over 10 years ago works with the latest releases. This has to be our benchmark to reach the same level of stability.

    And stable APIs doesn’t only mean stable PHP API. It is the same for our JS code or other changes that we might do in CSS, icons or other areas that might effect how third party apps work. If we do a change in the ownCloud core that breaks an older third party app then this is our fault and not the responsibility of the third party developer to constantly adapt the app to the core API changes. Our code checker is a good way to make sure that no private APIs are used that might break the app in an upcoming release. But not everything can be checked automatically. We have to keep this in mind when we develop the core and the APIs.

    When we have to make changes to the API, it is important to announce them in advance and keep the ‘old’ API still available rather than entirely replacing it. A new API should thus be introduced alongside the old API, which is then deprecated.

    I see three years as the sweet spot before we can remove a deprecated API, so developers and users have enough time to migrate. Of course, security can be an exception after careful considerations. It can also be helpful to create tools that help developers migrate to a new API.

    Of course, apps which use private APIs of ownCloud will simply have to migrate to public API. If a developer needs to use private APIs, we have to create a public API for him or her to use! App developers can discuss API needs on our development list or in github.

    Documentation

    Not all developers are deeply into ownCloud code and super experience developing apps. To aid app developers, our APIs need to be designed to be simple to use. Developer documentation is also extremely important. We can’t expect that third party developers read code from core or other apps to learn how a specific API has to be used or how something has to be implemented. So we have to provide high level developer documentation. Additionally tutorials, example apps and all kind of other resources help developers get up and running.

    User Communication and Promotion

    The third main principle needed to build a large app ecosystem is communication. We need to explain to both developers and users that we’re serious about apps, how they can get or develop apps and where to find help. We have to promote apps and manage expectations.

    Promote apps

    Third party developers need our help to promote their apps. We have to talk about them and blog about them to make them successful. Every ownCloud third party app should have the visibility that it deserves. One thing that ownCloud can do to help here is our app marketplace / app store at apps.owncloud.com. This makes it easy for third part developers to publish their app and make them available to all ownCloud users. The more popular these third party apps are the better for everyone in ownCloud.

    Manage expectations

    Not every third party app has the same level of quality, security and user experience. Instead of just rejecting low quality apps we have to explain to our users what to expect from which app. This also helps the developer of third party app developers to improve their apps to reach a higher quality level.

    These levels are partly reflected in the user ratings from the app store. But additionally we want to introduce different quality levels for apps which we actively triage. Apps have to comply to certain quality qoals to to be on a specific quality level. We will announce the changes that we introduce in ownCloud 8.1 here soon.

    Conclusion

    The key to building an ecosystem is taking the needs and requirements of third part developers seriously. If the development documentation is not good, if an API is missing or broken, if a questions from a third party developer is not answered fast enough then we as core developers have failed. It is the responsibility of all of us to always remember that.

     

     

    ]]>
    http://karlitschek.de/2015/06/building-a-platform/feed/ 11
    Scaling http://karlitschek.de/2015/03/scaling/ http://karlitschek.de/2015/03/scaling/#comments Wed, 25 Mar 2015 08:36:05 +0000 http://karlitschek.de/?p=510 I’ve visited both FOSDEM and SCALE over the last weeks, where spoke with dozens of people and gave talks about ownCloud 8. We’ve been getting a lot of positive feedback on the work we’re doing in ownCloud (thanks!) and that has been very motivating.

    Does it scale?

    A question which comes up frequently is: “What hardware should I run ownCloud on?” This sounds like a simple questions but if you give it a second thought, it is actually not so easy to answer. I had a small cubox on the booth as a demonstration that this is a way to run ownCloud. But development boards like the Raspberry Pi and the cubox might give the impression ownCloud is only suitable for very small installations – while in reality, worlds’ largest on-premise sync and share deployment has brought ownCloud to 500,000 students! So, ownCloud scales, and that introduces the subject of this blog.

    If you look up the term scalability on wikipedia you get the explanation that software scales well if you get double the performance gain out of it if you throw twice the hardware at the problem. This is called linear scalability, and rarely if ever achieved.

    The secret to scalability

    ownCloud runs on small Raspberry Pi’s for your friends and family at home but also on huge clusters of web servers where it can serve over hundreds of thousands of users and petabytes of data. The current Raspberry Pi doesn’t deliver blazing fast performance but it works and the new raspberry pi 2 announced last month should be great hardware for small ownCloud deployments. Big deployments like the one in Germany or at CERN are usually ‘spread out’ over multiple servers, which brings us to the secret sauce that makes scalable software possible.

    This secret to building great scalable software is to avoid central components that can be bottlenecks and use components that can easily be clustered by just adding just more server nodes.

    How ownCloud scales

    The core ownCloud Server is written in PHP which usually runs together with a web server like Apache or ISS on an application server like Linux or Windows. There is zero communication needed between the application nodes and the load can be distributed between different application servers by standard HTTP load balancers. This scales completely linear so if you want to handle double the load because you have double the users, you can just double the number of application servers making ownCloud perfectly scalable software.

    Unfortunately an ownCloud deployment still depends on a few centralized components that have the potential to become bottlenecks to scalability. These components are typically the file system, database, load balancer and sometimes session management. Let’s talk about each of those and what can be done to address potential performance issues in scaling them.

    File system scalability

    The file system is where ownCloud has its data stored, and it is thus very important for performance. The good news is that file systems are usually fast enough to not slow down ownCloud. A modern SSD, RAID setup, NFS server or Object Store can deliver data rates that are a lot faster than the typical internet network uplinks so you rarely bump into limits with data storage. And if you do, there are solutions like GlusterFS which help you scale performance quite easily. On the App server, a properly setup and configured Temp directory is important to achieve good performance as data has to flow via the ownCloud installation to the user (sync clients or web interface).

    Sometimes, this isn’t enough. If you have to handle petabytes of data, ownCloud 8 offers a solution developed together with CERN. This solution lets ownCloud act as a storage broker to direct read and write requests directly to the storage node where the data resides. The result is that that no actual payload flows through the ownCloud servers anymore and the storage performance is entirely dependent upon the data storage itself. Thanks to this solution, the file system storage should never be the bottle neck for ownCloud.

    Database scalability

    ownCloud uses a database to store all kind of metadata so it depends on a database which is very fast and scalable to keep performance up. ownCloud can use enterprise databases like MSSQL and Oracle DB which offer all kinds of fancy clustering options. Unfortunately they are proprietary and not necessarily cheap. Luckily there are Open Source alternatives like PostgreSQL and MySQL/MariaDB which also offer impressive clustering and scalability options. Especially MySQL combined with a Galera cluster is a very nice and fast option that is used by a lot of the larger ownCloud installations.

    Note that scalability doesn’t always mean big! Scalability also means that ownCloud should run fine on very tiny systems. Some embedded systems like the first Raspberry Pi had very limited RAM. In such situations it is nice to use SQLite which is embedded in the PHP runtime and has a very tiny memory footprint, saving precious system resources. This is is all about choice for the right system size!

    load balancer scalability

    If you have more than one application server than you need a way to distribute the incoming requests to the different servers. ownCloud uses a standard protocol like HTTP so that off the shelf solutions can be used for load balancing. There are standard and enterprise grade appliances from companies like F5 that are very fast and reliable if used for redundancy with a heat beat. Nowadays there are also very good and affordable options like the Open Source HAProxy on top of a standard Linux system available. This also works very well and is very fast. If you really have a lot of traffic and don’t want to buy hardware appliances you can combine several redundant HAProxy servers with DNS round robin. This has to be done very carefully so that you don’t compromise your high availability. There are several blogs and articles out there describing how to set up a system like this.

    Session management scalability

    There are two fundamentally different ways to do session management which are both supported by ownCloud. One is local session management on the application servers. The other is a centralized session management. I don’t want to discuss the pros and cons of both solutions here because they are a lot of aspects to consider. But with regards to scalability I want to mention that the simpler solution to have local session management together with sticky connections has the benefit that it does not need a central component. This means that it provides linear scalability. If a centralized session management is used then something like memcached is recommended and supported by ownCloud because it can also scale easily internally.

    summary

    ownCloud has been designed to scale from tiny embedded systems like a Raspberry Pi for a few users to a standard Linux server for a small workgroup to a big cluster for several hundred thousands of users. A wide variety of solutions and technologies can be used to make this possible and if you are interested in ways how to do this than have a look at the owncloud documentation for more information and look at the third party resources and white papers available for this on owncloud.com

    ]]>
    http://karlitschek.de/2015/03/scaling/feed/ 2
    2014 and 2015 http://karlitschek.de/2014/12/2014-and-2015/ http://karlitschek.de/2014/12/2014-and-2015/#comments Wed, 31 Dec 2014 12:03:38 +0000 http://karlitschek.de/?p=503 In just a few hours 2014 ends. This is a great opportunity to look back at what happened this year in the ownCloud world and in my personal life. This was an absolutely crazy 12-month so this blog post is now way longer that I planed. A huge thanks you to everyone in the ownCloud community. It´s a blast to work together with so many clever and friendly people from all over the world.

    ownCloud 7
    First and most importantly we released ownCloud 7. This was a major milestone in the history of ownCloud because of several reasons. We made significant improvements in the area of quality and overall polish. The user-interface was refreshed and we implemented a ton of improvements all over the place. You can have a look at the feature page if you are interested: http://owncloud.org/seven/

    But ownCloud 7 was also the first release which featured a big and strategic new direction for us which is server 2 server sharing. This is the first step to build a truly distributed and federated and open cloud. In my opinion it´s the future of a self controlled and self hosted cloud world, which respects privacy. But also enabled easy collaboration and sharing. ownCloud 8 will be another big step forward in this direction but this is something to discuss in 2015.

    ownCloud Inc.
    ownCloud Inc., the company behind ownCloud, grew significantly in number of customers and revenue. We opened up several new offices and hired a lot of new employees. This is very exiting for me because it´s so much fun to work with so many smart and dedicated people on ownCloud. I´m super happy to see how employees and community volunteers work together to push ownCloud forward.

    ownCloud community
    The ownCloud community is the source and foundation of the ownCloud development. I´m super happy that we are growing significantly and that more and more people join us to drive ownCloud forward. We use Bitergia to measure some community metrics. You can look at the statistics here:  http://bitergia.com/projects/ownCloud-dashboard/browser/  Jos blogged about this last summer and we had close to 300 contributors in the past 12 month at the time. http://blog.jospoortvliet.com/2014/08/owncloud-numbers.html This is a very impressive number and ownCloud is already one of the biggest open source projects out there. But if you look at the statistics today than you see that we already grew to 372 contributors in the last 12 month since August in the meantime. This is just amazing. So I´m sure that everyone who wants to contribute to ownCloud finds a very friendly and open community to join. Let´s make the world a bit better together.

    Lately we launched the new ownCloud Meetup program. Everybody is invited to start local ownCloud meetups and get-togethers.
    We provide a lot of  useful helpful information on our website and you will get support if you want to organize an ownCloud event in your area. More information here: http://owncloud.org/events/meetups/

    We also launched the ownCloud insiders program if you want to help to promote ownCloud https://forum.owncloud.org/viewtopic.php?f=8&t=24661
    The ownCloud community will be at FOSDEM and SCALE next year with tables. So if you are at one of the events please stop by and say Hi :-)

    In the past a lot of people asked about the license of the ownCloud iOS app. So I´m super happy that a few weeks ago we found a way to open source the app. So everyone who want´s to contribute has now the opportunity: https://github.com/owncloud/ios

    ownCloud contributor conference
    Last summer we organized our yearly ownCloud developer meeting in Berlin again. Because of our growing community of contributors we were able to transform this into the very first ownCloud Contributor Conference. Over 100 contributors joined to listen to talks, discuss, write code and socialize. You find some impressions here:
    https://www.youtube.com/watch?v=ZNbBeXyE9fw This was a crazy event. Especially if you remember that the very first ownCloud meeting in 2010, just four years ago, had only 5 attendees.

    Awards
    A few days ago I learned that the Linux Journal readers voted ownCloud as number 4 of the best new open-source projects http://www.linuxjournal.com/rc2014?page=21 and number two of the best cloud based file storage solution. http://www.linuxjournal.com/rc2014?page=14 This is very exiting because only Dropbox is more popular at the moment that ownCloud.  The gap is shrinking fast compared to the numbers of last year.
    A few days ago I learned that I won the Stiftung Neue Allmende award for starting and maintaining ownCloud which is endowed with 550,- EUR. This is a huge honor. Thanks a lot for that. Obviously ownCloud is not only developed by me but by our community. So I will donate this to the ownCloud community. We will use this for a nice social event at the next Contributor Conference next summer. Yay! Thanks a lot again :-)

    Personal
    2014 was also a special year for me personally. I moved to Boston, MA last spring to be able to work closer with my colleagues in the ownCloud Inc. headquarters in Boston. This is definitely fun and an interesting experience to live in the US full-time. I also took the opportunity to buy a motorcycle again which is awesome. I have a lot of fun with discover the great nature in New England by bike.

    I also had the opportunity to meet very interesting people like Tim Berners-Lee, Linus Torwalds and RMS to discuss ownCloud and it´s great to hear that they all like it. What is also very exiting for me is that we are also working closely together with very clever people from the W3C offices at MIT Cambridge and CERN in Switzerland to develop innovative technologies that will hopefully land in ownCloud next year.

    I also did a ton of travel last year to give talks and keynotes at conferences all over the world. Most important for me was the invitation the give a keynote as LinuxCon Europe about the future of cloud computing and federated clouds. This was a good opportunity to share my vision about distributed and federated clouds with a broader audience.

    2015
    So what are the plans for next year?
    The next big milestone is obviously ownCloud 8. Feature freeze is today the Dec 31th with a planed release date of early February. ownCloud, the software and the contributor community, reached a size where big monolithic releases don’t work anymore. The release of 7 was already very hard to coordinate, stabilize, test and to document all the features. So in the future we want to reduce the size of the core ownCloud server again and also release it more often and move to a time based release schedule.

    So starting with ownCloud 8 we will release a major new ownCloud server version once a quarter. The apps on top should be compatible between different server versions and will be partly released independently.

    On January 17th is the 5th birthday of ownCloud. At that day I announced the ownCloud initiative during the CampKDE keynote in San Diego. A bit later I released the first beta version and the first contributors joined. It’s amazing to see what grew out of a crazy idea. Does someone has an creative idea how to celebrate this special day? Please let me know.

    Another important topic for next year will be the launch of the user data manifesto 2.0.  Hugo , Jan and I are working on this and a first draft is already out. https://userdatamanifesto.org/2.0/
    We got a lot of feedback from smart people and organization that we already incorporated. Everyone is more than welcome to participate in the discussions.

    Thanks to everyone who made 2014 such a great year. I´m looking forward to keep on working with awesome people in 2015 again to push ownCloud and the ideas behind it forward. Let´s make the world a little bit better together.

    Thanks and take care in 2015
    Frank

    ]]>
    http://karlitschek.de/2014/12/2014-and-2015/feed/ 3
    Ohio Linux Fest http://karlitschek.de/2014/10/ohio-linux-fest/ http://karlitschek.de/2014/10/ohio-linux-fest/#comments Wed, 22 Oct 2014 21:52:25 +0000 http://karlitschek.de/?p=493 This Friday the Ohio Linux Fest kicks off in downtown Columbus and I’ll be there! It is my first Linux Fest in the US so I greatly look forward to being introduced in this grand tradition. Of course, I’ll be talking about ownCloud on Saturday the 25, from 13:00 to 14:00 room D142-143.

    The title of the talk is “crushing data silos with ownCloud”: helping people liberate their data from the centralized services they have stored it on. I don’t think that a world where most of the personal data of the world is stored on servers of a hand full companies is a good one. ownCloud is, right now, the best way of getting out of that world!
    The talk will also cover a few interesting new ideas that we want to do in ownCloud to build a fully federated and distributed solution in the future.

    It would be awesome to do a small ownCloud and/or KDE meetup there.

    If you’re going to be there – let me know! Find me on twitter and lots of other social services!

    ]]>
    http://karlitschek.de/2014/10/ohio-linux-fest/feed/ 0
    A possible future for PHP http://karlitschek.de/2014/10/a-possible-future-for-php/ http://karlitschek.de/2014/10/a-possible-future-for-php/#comments Thu, 02 Oct 2014 11:14:38 +0000 http://karlitschek.de/?p=484 ownCloud is one of the biggest open source project written in PHP if you look into the latest statistics. It is used for the server part of ownCloudas most of you know. We use other technologies like C++ and Qt for the Desktop Clients, Java for the Android app and Objective-C for iOS, JavaScript for the web-interface and more. But the heart of ownCloud is the server component which is using PHP 5.3 or higher..

    There were several reason for choosing PHP:

    • The mission of ownCloud is to enable everybody to host their own cloud server. PHP is the technology that is available on most webservers, operating systems and platforms. So we make hosting of an ownCloud server super easier because it is written in PHP.
    • PHP is a scripting language which means that one server tar file runs on all platforms and no complicated cross-compiling and packaging is needed
    • PHP is very well known. A lot of people are familiar with PHP. And even the developers who don´t know PHP can learn is relatively easy. This is very important especially for an open source project. The bar to become a contributor should be as low as possible.
    • PHP is fast and quite powerful if used in the right way. A lot of big web application like Wikipedia, Facebook, WordPress and parts of Yahoo are written in PHP. So you can do a lot with it. Unfortunately is is also relatively easy to write bad PHP code. But more about this later.
    • There is a huge ecosystem of libraries, components and connectors/drivers available for PHP. For an open source project like ownCloud this is very cool because this means that you don´t have to reinvent everything from scratch. We stand on the shoulders of giants.

    PHP is not the most hip programming language in the world. Actually the opposite. It has a relatively bad reputation. I personally was never a big fan of choosing the technologies based on what is cool or “modern” or in vogue. I think there are different technology for different jobs and they should be evaluated objectively and choose without to much emotion involved. So I don´t understand the religious discussions why tool x is always better than technology y. I think it is all about the right technology for the job after a fair assessment of course.

    So i´m still very happy with this decision to use PHP. So far we have not seen any bigger architectural technical problems that we can´t solve with PHP.

    Does this mean that PHP is perfect and I´m super happy with everything? Of course not. PHP was developed in the mid 90s at a time where no one could have imagined how the web looks like today. Some of the cool features of the time turned into a nightmare today. There is a lot to improve and I think even the core PHP developers agree with me here.

    A few of the obvious shortcomings are:

    • Security. PHP in itself is not insecure and it is obviously possible to write perfectly fine and secure applications with PHP. But PHP decided to implement an quite naive approach about security and doesn´t support the developer too much in writing secure code. To be fair everybody was naive about web security in the 90s. So there are a not a lot of features available in PHP that actively support you with writing secure code. The database situation is a mess so a lot of people still don´t use prepared statement which leads to possible SQL injection. And filtering incoming data for XSS and other problems has to be done relatively manually. There are extensions and libraries available to help with all this problems but they are not part of the language/runtime core or are incomplete.
    • compile time / runtime configuration. Just for fun call the ./configure script to compile php yourself and look at all the compile options. And now look at all the options that can be set in php.ini by the server admin. On one side this is cool because an admin can enable and disable a ton for core features in PHP in a very fine granular way. But as a developer of an PHP application that should run on all available PHP servers this is a nightmare. You never know which feature is enabled and available. In ownCloud we have a lot of code that s the environment and the runtime to see if everything works as expected and adapts to it as needed. This is unfortunately not what you call a stable platform and a good OS abstraction.
    • There are some inconsistencies in the function and class namings. Sometimes unerscores are used and sometimes camel-case. Some features are available in a procedural style and some have an OO API and some even have both. There is a lot that should be cleaned up.
    • Static typing. This is totally a question of taste but sometimes I would really love to have a bit more static typing in PHP. Guess what this following code does if you have a file named “0” in your directory

    while ( ($filename = readdir($dh)) == true) $files[] = $filename;

    I would really love to see PHP moving to the next level and improving some of this shortcomings because most of it is really good.
    But it is very important to do it right.

    A latest article at ArsTechnica and Apples move to introduce Swift as Objective-C successor triggered my fantasy how a next generation PHP could and should be done. Keep a programming language backwards compatible or fix its flaws? – Apple Swift

    There is the old and to be honest quite naive approach. The core team of a programming language just releases a new and incompatible version that fixes the flaws of the older version. Examples are Perl or Python. The problem is that it´s close to impossible to rewrite a big software project to make it compatible with a new version. So you end up with two versions of the programming language/framework/runtime for a very long time. Some applications run on the old version and some run on the old version. Libraries and dependencies are sometimes only available for one of the versions.
    Migration is super hard and can´t be done piece by piece. Please see Perl6 and Phyton 2/3  as examples what a nightmare this can be. Both exist for a very long time and a lot of software is stuck in the middle of a migration story somewhere.

    A more positive example is C++. It ´s still a very different language than C but the good thing is that it can be mixed inside an application. So in the 90s C developers were able to use the cool new C++ features in one part of the application without the need to rewrite everything from scratch.

    Apples move to introduce Swift as a successor of Objective-C is very clever in my opinion. It´s completely new language but it´s running on the same runtime. This means that a developer can take an existing Objective-C application an just start to write the new features in Swift or replace pieces one after another with new Swift code. This than compiles into one binary that has no new runtime dependencies compared with Objective-C.

    I wish PHP would do something that makes it possible to evolve and improve the language significantly but still provides a smooth migration experience not like Perl and Python did with introducing completely new backward incompatible releases.

    So a good solution would be if PHP 6 or 7 would introduce a new tag to start a php file. For example <?PHPNEXT instead of <?PHP. Both modes are fully supported by the new PHP version and can be used in parallel in the same application or even in the same file. In the NEXT section the new and improved syntax is used.

    Here are a few ideas for improvements that I would love to see:

    • Security. Kill the _GET and _POST and _SERVER arrays and introduce a proper API that can be used to filter all incoming data.
    • Database. PHP support a ton of different database API. Some of them are very old but they are inconsistent to use. Everything should be standardized so that only one OO interface exists. I personally would use PDO as a starting-point here.
    • 32bit / 64bit. Anyone who ever tried to write a PHP application that runs on 32bit or 64bit operating-systems will recognize that variables especially integers behave differently. I understand that this is a reminiszense to C/C++ but this is seriously a bad idea. I don´t want to have different code paths which have to be tested independently.
    • kill save_mode, open_basedir and other acient concepts
    • Remove most of the compile and runtime config options. All PHPNEXT runtime environments should be as similar and stable as possible.
    • Typing. It would be cool if PHP would introduce optional static typing. So that a variable can be declared as, for example, bool or int. An exception should be thrown if used otherwise.
    • Always use unicode strings

    Some of this improvements are implemented in Hack which is some kind of PHP fork developed by Facebook. Hack is indeed an interesting concept that goes into a similar direction. They also use a new tag “<hh” so that code can be mixed in one file and they improve typing. At the moment it´s not clear how much energy Facebook will invest in the future to push Hack forward and how much adoption it will get outside Facebook. I´m especially worried how open they are for changes that are not important for them, how well and open this is governed. I would prefer an official and more generic approach from the PHP community which will be part of one of the next main PHP releases.

    I hope by dream of a more modern and cleaned up PHP including a smooth migration path becomes reality in the next few years.
    Obviously we at ownCloud couldn´t start to migrate to this new PHP mode before 95% of all PHP installations out there run with the new version. This will easily take additional 3-5 years.

    By doing this big projects like WordPress or ownCloud would actually have a realistic chance to move to a cleaner and more modern language. But more importantly this would make PHP ready for the challenges of the future.

    Please leave a comment here in the blog if you have an opinion about this.

    ]]>
    http://karlitschek.de/2014/10/a-possible-future-for-php/feed/ 67
    ownCloud 7 is out! http://karlitschek.de/2014/07/owncloud-7-is-out/ http://karlitschek.de/2014/07/owncloud-7-is-out/#comments Wed, 23 Jul 2014 13:06:16 +0000 http://karlitschek.de/?p=461 ownCloud 7 We just published ownCloud 7!

    This awesome release brings many new features. Among them, I’m most excited about the server to server sharing.

    Server to server sharing is a first step in true federation of data with ownCloud: you can add a folder shared with you from another ownCloud instance into your own. Next step would of course be to also share things like user accounts and data like chat, contacts, calendar and more. These things come with their own challenges and we’re not there yet, but if you want to help work on it – join us for the ownCloud Contributor Conference in Berlin next month!

    A close runner-up in terms of excitement for me are the improvements to ownCloud Documents – real-time document editing directly on your ownCloud! We have been updating this through the 6.0.x series so the only ‘unique’ ownCloud 7 feature is the support for transparently converting MS Word documents, but that is a feature that makes Documents many times more useful!

    There are many more features, you can find more details on the ownCloud website. The official announcement blog post is here.

    A big THANK YOU

    This would not have been possible without the hard work of the ownCloud community, so a big thank-you goes out to everybody who contributed! We have a large team of almost 100 regular contributors, making ownCloud one of the largest Open Source projects and that makes me proud.

    Of course we have a lot of work to do: revelations of companies and governments spying on people keep coming out and our work is crucial to protect our privacy for the future. If you want to help out with this important work, consider contributing to ownCloud. We can use help in many areas, not just coding. Translation, marketing and design are all important for the success of ownCloud!

    The release of ownCloud 7 is not only the conclusion of a lot of hard work by the ownCloud community, but also a new beginning! Not only will we release updates to this release, fixing issues and adding translations, but the community now also starts to update the numerous ownCloud apps to ownCloud 7.

    Expect more from us. Now, go, install ownCloud 7 and let me know what you think of it!

    ]]>
    http://karlitschek.de/2014/07/owncloud-7-is-out/feed/ 14
    Free ownCloud mobile libraries released http://karlitschek.de/2014/02/free-owncloud-mobile-libraries-released/ http://karlitschek.de/2014/02/free-owncloud-mobile-libraries-released/#comments Thu, 20 Feb 2014 13:20:00 +0000 http://stage.karlitschek.de/2014/02/free-owncloud-mobile-libraries-released/ Today we are happy to announce the release of the ownCloud mobile libraries for iOS and Android. ownCloud is a free file sync and share solution. The main differentiator to Dropbox, Google Drive, and others beside being free software is that you can run it yourself wherever you want. Obviously a central place to you store your files is only useful if you can access it from all devices and integrate it with all of the applications that you use.
    Because of that, the ownCloud strategy is to provide as many ways as possible to access files and data stored in ownCloud. We have a strong commitment to support open protocols and formats like WebDAV, CalDAV, CardDAV, OCS, ODF and others. So you can mount your ownCloud via WebDAV easily with KDE, GNOME, Windows, Mac and so on. It’s easy to integrate ownCloud with other systems by transferring files via WebDAV. This is the power of open protocols. But ownCloud provides much more than that. There are mobile applications for iOS and Android and Desktop syncing
    clients that you can use to work with you files. The Desktop syncing clients runs on Mac, Windows and Linux and also ship with a command line client that can be used to automatically sync folders between desktops and the server or script it in any way. There is also a C++ library that can be used by 3rdparty clients like the KDE Plasma one. So there are a lot of options to access your ownCloud from the desktop.
    On the phone and tablet side it was, until now, a bit more difficult. A user could use the official ownCloud apps, but if a 3rd party app wanted to access an ownCloud server, then the 3rd party app had to implement all of the WebDAV and REST calls needed to talk to ownCloud.
    This is why today we released free libraries for iOS and Android that can be used used by mobile developers to add ownCloud support to their apps. They provide easy to use methods to read and write files, share files and many more useful operations. To make these libraries as useful as possible to as many developers as possible, we have released them under the MIT license.
    The libraries can be download here
    The documentation how to use them can be found here:
    For any developer related questions please post to the new ownCloud developer list http://mailman.owncloud.org/mailman/listinfo/devel

     

    ]]>
    http://karlitschek.de/2014/02/free-owncloud-mobile-libraries-released/feed/ 6