OBS source repository moved

The OBS git repos have been moved to github.com as announced earlier on build service mailing list. Please read this mail for details.

Advertisements

Policy proposal for Factory: Make source of tar balls trackable

I like to suggest a general policy for openSUSE:Factory project to document from where a tar ball (or any other file from upstream) is comming from. Why that ? It makes it much easier to review version updates and it guarantees that no one can inject some mal code via a modifed tar ball.

So far I added the source services “download_url” and “tar_scm” to our OBS instance, which downloads the files and stores them as files via a commit. Some people use them already, some others don’t like them because they store the files with _service: prefix.

In last hackweek, I added another way to handle this, which I would like to request as setup and policy for openSUSE:Factory project. You can add a project wide source service, for example the new “download_files” service. That would mean that no needs to add a _service file to the sources anymore. It is enough to add an URL to the spec file Source: tags. The service will automatically download it from there.

But that does mean we still have have _service:download_files:osc-0.1.tar.bz2 file names ? Not when we also add the new “trylocal” parameter and use latest osc versions. This parameter will let act osc to execute the services, but name the files without prefix and commit them together with the other files.

Where is the advantage then ? The server is still validating that this is an identical file. It downloads it again and compares it. In case it is the same file, nothing will happen.

What will happen, when the file differes ? We basically have two options, either we can let the service mark the source as broken or we would store the file with _service: prefix again.

The later mode has the advantage that you can still do version upgrades via slow connections and let the server download the files.

Please find some more details about new possibilities with the source services here.

An example setup for this can be tested via

osc bco home:adrianSuSE:FactoryTest bc

and do for example a version downgrade to 1.05 version to see how it works. Please note that you need the osc from openSUSE:Tools:Unstable project for this.

We can also apply the still suse-internal spec formater and validator scripts via this way later one.

Another advantage of this setup would be the new “update_source” service, which could run in some openSUSE:Factory:AutoUpdate project and tries automatic version upgrades when upstream releases a new version. They could be reviewed and just picked (directly or with additional manual fixes).

How to work with OBS via slow connections

The openSUSE Build Service Books have a new chapter called HOWTOs. These will be written for asked questions, the first one existing is how to work best with OBS if you have limited bandwidth. There are multiple methods to avoid large downloads or even uploads of large files (like tar balls) and let just the server do the work…

Fotostream from openSUSE Conference 2010

Yet another foto stream from the openSUSE conference. You see the desktop leads from KDE and Gnome (Cornelius Schumacher and Vincent Untz) giving a talk about the past and future of the free desktop, Stephan Kulow about the future of the distribution, Bernhard Wiedemann about QA testing and so on.

Most important may be the presentation of the openSUSE board (mainly by Pascal Bleser) how they plan to found an independent foundation for openSUSE as non-profit organization. An important rule of that foundation is that it is independent of any company (no majority of Novell here) but can handle sponsoring, partnering and trademark questions.

We had also very filled rooms during the OBS talks, but I was unable to take pictures at that point of time unfortunately 😉

OBS Development Team Member Job Position

SUSE GmbH has currently a job position open for an OBS Developer. Find details on the job position page at Novell.

OBS is used in the openSUSE project, but also internally at Novell and at plenty other places and companies.

The downside will be of course that you will have to work together with people like me 😉

Some LinuxTag 2010 impressions

LinuxTag 2010 has ended, openSUSE had a booth in the community area and we had a number talks. We also released OBS 2.0 on LinuxTag. You know this of course already, but here are some impressions.

openSUSE booth was very well visited. Various workshops and activities created several times actually a big swarm around it. Many people were interessted about OBS in special and I hope we won some more OBS users and developers.

Hennes and mine talk about “how to escape the free software hell” was provocant enough to get quite some people into our room directly after the keynote. I hope we were able to show off the coolness of OBS there.

Read the comments in the picture gallery for some background information.

OBS reports scheduling state now

After getting asked daily why package X or project Y is in a certain state, why it is not yet published or if a “trigger rebuild” button click is needed, I finally got around and implemented the last planned feature for OBS 1.7.

Each repository has a scheduler state now for each architecture. This state tells you in which state the scheduler saw it the last time he looked at it. In addition to that, there is also a “dirty” flag. This gets set, when something happened what requires a re-calculation by the scheduler, but he has not done this yet. So no need to hit the “trigger rebuild” button in this case, it won’t help 😉

A typical state round trip is the following, it can stop at any point and restart from a higher listed state again if this is needed:

unknown: a repo has just been created for this architecture. The scheduler has not seen this yet. This is currently the case also in most repos, just because the scheduler with this new code has not looked at all existing repos yet. But you will see this only in very rare cases in future.
scheduling: the scheduler is looking at this repo right now. The package states will get updated soon (from 1 second up to 2 minutes, depending on the project complexity)

blocked: packages from other repos need to build first until any of the packages from this repo can get build.
building: packages are scheduled for building, this means they are in “build”, “scheduled” or “finished” state.

unpublished: all packages got built, publishing is disabled, so nothing is to do here anymore.
or
finished: all packages which require a build have been built. This repo is enabled for publishing, but this has not yet happened.
publishing: The publisher is currently processing this repository.
published: the latest built packages have been uploaded to the stage server and are available via download.opensuse.org

Future plans for this are to add this state list in the web interface somehow, so everybody gets an explanation in the context.

Another thing what I want to add is a button beside the “unpublished” state, to force a publishing. So by default nothing gets published, but the project owner can force it when he things it makes sense. However, this is now also possible to achieve by swithing the publish flag and revert it after some time. The OBS is now even publishing when the build has already finished (unlike before).