Social Help for Sugar: Week 7

A little over seven weeks of the GSoC coding period is gone. Fortunately, I’m mostly done with the work excepting a few things to polish up and then we can work to actually deploy the social help activity . This is a good thing since the autumn semester at my college starts in just four days and I might not be able to dedicate too much time to the project.

Anyway, this week was hectic – I had to get a ton of things in order for the new college year. Nevertheless, I did manage to get some work done.

First, I went through a mini heart attack when the inbuilt database backup thingy didn’t work on my local machine. It was weird, really. Everything had worked before when it just suddenly stopped accepting backups. After several less than successful attempts to get the backups to work, I just gave up on my local machine and decided to try creating a shiny new docker instance and try the backup on it instead. And much to my relief, it did finally work.

Then, I wrote to Paul, explaining everything we’ve accomplished till now. I also asked whether it was high time for a code review. While I waited for the reply, I decided to see if I can update discourse to the latest version. Normally, the admin would use the inbuilt option in discourse to do this but of course, we couldn’t do such a thing since we’re using a custom version of discourse. So, I manually did a fetch-and-merge on my github repository of discourse, created yet another docker instance and installed the latest version on it. It, too, worked flawlessly. Come to think of it, I’m liking docker more and more.

Anyway, Paul replied to me soon after. He wanted us to have a separate button to launch social-help for every activity along with the shortcut thingy. So, I talked on the IRC, got some clues and added the button. It works, now.

That’s all for this week.

Social Help for Sugar: Week 6

Another week has gone by and my project is mostly working. Last week, I had worked on deployment and after a lot of grief, I was able to able to get a Docker instance running. This week, I had planned on working on database migration scripts – which I did, eventually.

The first thing I did this week was to talk to Paul, asking him about the server space requirements that we’ll have at the time of launch. Paul connected me to Matthew who’s a sysadmin and we’ve discussed some of the details since then. Matthew told me that they’ll have a meeting soon to discuss getting a new VPN for the official deployment.

Then, I decided to clean up a few things. Until now, I had hard-coded the address of the forum. However, that would never work in practice, specially because we’re aiming to get a social-help instance running in every non-internet area. So, I decided to add an option to the control-panel, much like the option for the jabber server. It took me a bit of time but finally, I got round to doing that after a few failures. Now, users can fill in a custom URL for the social-help discussion forum.

The next thing, and a very important thing, was the db migration scripts. The idea is that we’ll have a central repository of information – the central discussion forum running on a subdomain of sugarlabs.org. This central forum will have most of the information. However, as I’ve stated before, we’re making things in such a way that the discussion forum can work even in areas with no connectivity, we’d like to be able to deploy discourse in these areas too. Deployment is pretty easy with docker, as I’ve written in my previous posts. However, these localized discourse instances will be devoid of any information. So, I’m thinking that it will be good to have a way to initialize these new discourse instances with all the information contained in the central repository. That’s why wee needed a db migration script.

Now comes the problem. Taking dump of the running postgres database is easy. The problem is the restoration. To restore the dump, we’ll need to stop the server for a bit. However, to stop the server, we’ll first need to log in to the running docker instance. But, for the docker instance to exist, we’ll need it running! And that’s the dilemma. We can’t have a running server to get a backup but if the we stop the server, the docker instance cannot be logged into. So, I was in a fix.

I tried a couple of things, all to no avail. I was despairing, really. Then, I though maybe the discourse devs had already thought of this problem – what with a new version of discourse coming out every week. And voila! They had thought of it after a few searches on meta.discourse.org, I hit jackpot. There’s a guide that shows exactly how to do this thing. So, I was saved again by the foresight of the discourse developers. Thanks guys!

This next week, I guess that I’ll get a final review from Paul and start working on a code review. Then, with some hope, Matthew will have the server space for me and we’ll probably be able to officially launch this thing.

Social Help for Sugar : Week 5

Another week is gone and this one was special – we (all student participants of GSoC) had our midterm evaluations this week. So, I needed to show the working discourse website to my mentor, Paul.

The aim of the project I am doing is to create a StackOverflow like platform for sugar where users and developers can easily collaborate together to create a knowledge database for the sugar desktop environment and its activities. Also, since sugar is often employed in areas without any connectivity, we need to make sure that social help is easy to set-up on a server. The problem however is this: Discourse is a Ruby on Rails app that requires several different services (sidekiq, postgres, redis …) to work effectively. Couple that with all the different tools and gems required by the ruby based runtime environment along with version issues and you’ve got a headache for even the best sysadmins. There’s just too much stuff to install – I had quite a few problems because of this (since by distro is somewhat old) when I first installed discourse on my local machine. Top this off with the usual deployment problems with any rails app and you’ve got a true headache on your hands (or in your head? Idk).

Discourse devs saw this, though. So, they use Docker based images to make deployment easy. Now, this option works like a charm – I tested it myself. However, since I had made a few changes to the development version of discourse, I could no longer install from the official image that discourse devs provide. So, I spent quite a bit of time reading over Docker – I swear it was like learning git all over again with a completely different architecture. Anyway, once I was sufficiently comfortable with docker, I decided to create a new image for my customized version. A few looks at the official docker installers for discourse disabused me of any such notions – creating a new image would be a very long task. So, it was best if I did something else. I decided to ask the discourse devs and a somewhat long discussion ensued. Initial responses were somewhat vague and it was I who ended up suggesting a possible approach – that of editing Dockerfiles which the devs concurred with. I tried that – I really tried it – to no effect. After a few variations, I was ready to tear my hair off my haid. Fortunately, the final response from the devs saved me from a bald head.

So, in the end, all I needed to do was to edit the pups templates, not the Dockerfiles. That I did. And, it worked! (Sure, I had a couple of 502 Bad Gateways to deal with on the nginx first but let’s not spoil the fun).

Then, there was a the problem of emails to be fixed. See, discourse is a highly email reliant system – it just won’t work well. So, I tried the tested route – I just used the google’s smtp server via my gmail account. Google, however, had different ideas than my own. I got a bunch of mails telling me of suspicious login attempts. I tried several times to dissuade Google of its protective notions of my online identity all to be left with more suspicious-activity mails. Meh! So I decided to use a local smtp server and to this end, I set-up postfix. It seemed to be working well for a few python scripts so said, what the heck… let’s just use this and be done with it. Alas, discourse seemingly doesn’t play well with postfix – the mails sent my discourse never even made it to the postfix server. I checked the logs only to be receiving blanks. What can I say except for this: Arghhh. So, I sniffed around a bit on meta.discourse.org to see what other people were using. The name Mandrill popped up a few times and I decide to give it a try. And voila! It worked. I could have danced if only I wasn’t sitting in a public tourist viewing spot on the edge of a spectacular cliff. Oh well.

So, I’ve got a method to deploy this custom version of discourse pretty easily and a working email. I mailed Paul and he’s also set-up an account over on the ec2 instance. Anyway, this just means a lot less work towards the end of the coding period when we’ll be working to deploy social help on the public sugar server. For this next week, I will start working on the DB migration issues and hopefully we’ll be able to come up with a simple solution for that as well. This will just mean that whenever we have to set-up a localized social help in areas without internet connectivity, we can just shift the entire main server’s information onto the newer server, no problem. See you next week!

GSoC Week 4: Social Help for Sugar

Another week is gone by. This week, I finished up the login popup that I had discussed with Paul. Also, I’m done with the custom CSS styling. It wasn’t much really – just making the website a little more colourful. A designer friend of mine in fact told me that the original design was ‘pretty good as it is’. Nevertheless, I’ve given the whole website a standard bluish look.

I’ve also made the social-help activity a little lighter – I had originally planned to take away both bookmarking capability and the tabbed view. However, better sense prevailed and I decided that tabbed view is a necessity if the user wants to open an external link side-by-side with the forum.

The next part I wanted to work on was the DB migration scripts – I suspect I’ll have to chat with the the discourse devs for this. We’d like to keep the central DB as it is and migrate it to any new, local setups. This will include all the everything. That means we’ll be keeping the user data along with the questions/discussion.

This is what I wanted to work on this week. But, seeing as how midterm evaluations are fast approaching, I decided to use my time working on deployment. Discourse is a ruby on rails app. Not just that, it  requires several other services (like redis, for example) going on the side. This means that the deployment of the app on a production server is going to be a hassle at best and a hair-tearing, why-won’t-you-work experience during the bad times. Fortunately, the Discourse team already foresaw this problem – they’re using Docker container to address this problem. Docker is kind of like a VM – except for all the overhead of the VM as it uses the OS’s kernel.

So anyway, this means that the deployment process is straightforward – at least for the standard Discourse images. This, however, is not the case with us. I’ve made some small changes to the latest version of discourse and thus, I need to get a docker image for my customized version of discourse, not the standard one. Creating docker images is, however, not at all clear to me at this point. I’ve read the docker documentation and I understand some of it – it’s kind of time git. Every change to a base image is added in the form of layers – kind of like a git commit. Still, the actual process of creating an image eludes me. After lot of rather depressing hours, I decided to ask a question to the discourse devs. The answers were rather half-hearted and rather succinct – I still have little idea how to go ahead. So, I’ve asked the devs to elaborate on their previous answers. We’re in opposite time zones and so, I hope to get an answer soon. I have, in the meantime, procured an AWS micro EC2 instance for online deployment. I’ve already tested it using a standard discourse docker image and it works like a charm. I hope we can get this to work as well and soon.

Plans for the next week include getting the custom docker image to work and getting the whole thing run smoothly on the ec2 instance. Once the midterm evaluations are done, I’ll move on the the migration scripts.

Social Help for Sugar: Week 3

Third week is gone and another few things have been done. Also, this post comes a bit late – sorry about that. Next one will be on time.

Now, getting on with things. This week began with a discussion on sugar-devel whether I should use a webservice interface with social-help. The discussion clarified to me what is a webservice used for; social-help not being a likely candidate for usage. So, I’ve decided that creating a webservice interface isn’t necessary – the social-help activity handles user sessions quite nicely.

Then, I tried injecting some javascript into the Discourse webpage via WebKit – just a couple of alerts – and they work just fine. This I did because I wanted to autofill the username field of the registration form. I tried doing this using jquery based events. That didn’t work – I have no idea why. The JS console shows no errors or warnings. I can speculate however. I suspect that the code doesn’t work because the Discourse website is entirely ‘web 2.0’ – it’s all ajax. The whole thing is built on top of ember.js and the whole of it is loaded in realtime. That means that the elements that I wish to select via the jquery selector isn’t even there most of the time. So, I suppose this thing might not work despite the best of intentions. I will give it another shot later on.

In the last post, I mentioned that I had a few doubts about how to handle the login sessions for multiple users. The simple solution was to have a Gmail like popup on the top which disappears on its own in a few seconds. This popup shows displays the name of the currently logged in user and prompts you to logout. I actually do not know Ruby on Rails at all. To this end, I asked the discourse devs about how to go on adding this popup. Well, that question didn’t bear fruit – I was given a link of an existing Discourse extension and told that I should make an extension to display the popups. Ah well. I had to study ROR for some time to get how things worked. Anyway, that works now. Maybe, I’ll make a Discourse extension of this. Maybe.

Anyway, I also tried another colour scheme – one that doesn’t use the standard OLPC color palette. I think this one’s a bit better. Also, Paul sent me a couple of screenshots of Discourse on an XO so that I’ll the the font size adjusted according to the high DPI screen on the XO.

This next week, I’m going to finish up with the new colour scheme and I’ll start stripping the browser of unnecessary things.

Social Help for Sugar: Week 2

So, another week gone. Here’s the report for this week’s work, though I’m not sure if it’s much.

Last week, I had made a working social-help activity that opened up to the forum. From there, I changed a few things so that pressing a shortcut opens up the activity to the respective category of the forum. This presented a bit of a problem – namely that of unique names for categories. We know that the reverse domain names of the activities are unique. However, these make crappy category name. But, something like org.laptop.Terminal as a category would probably confuse the user quite badly. So, we needed some kind of map. Therefore, I’m using a Python dictionary that maps the activity’s unique name to a category name. In the rare case that the category name isn’t unique, we can map the newer activity to a somewhat longer name, I suppose. Anyway, this is a problem solved.

The next thing on my list was to get a way to autofill the registration form details from the sugar shell. For this, I poured over gtk WebKit documentation (which, to my chagrin, is quite scant for PyGTK and so I had to read docs for C++ based GTK) for the better part of a day with no luck. So, I decided to Google a way out of this. No luck there either. If anything, one thing was made clear – WebKit is just a page renderer – there is no way to control what is being rendered. And so, another hopeless pursuit came to a rather disappointing end.

Then, I rather unspectacularly took a 3 day break. More like the break took me. So, as things are, I’m visiting Hyderabad right now and there happens to be a great dam nearby. And my dear friends just had to drag me there with them to the comforts of an absolutely gentle heat of a 45 degree (celsius, mind you) sun in the middle of nowhere. Oh well. I guess I should be thankful I didn’t burn down.

Moving on. So, the next task to be done was to find a solution to the login problem. This I discussed with Paul. Let me quote the mail:

Me: The other thing I wanted to do isn’t entirely clear to me either. I’ll try to describe it though. Now, sugar is used in the XOs and it will be often just one user that uses a laptop. This means that once a user logs in the first time, he should never really need to log in again.

Paul: Sort of true. When you’re dealing with schools and classroom environments you’re probably going to some measure of laptops getting swapped around. For instance, at the end of the year will each child take their XO into their next classroom – or will the XO stay at the classroom and the kids use different XOs each year? It’s up to the school to decide how they handle it, and they might do it differently. Alternatively, there might be a school with 250 students and 50 XOs (two classrooms) – and classrooms share the XOs, class A gets them for a month, and then class B gets them next month

Me: Now, of course the browser will do this automatically unless the user clears the cookies or logs out. As far as I know, there is no option to clear cookies through the browser (the social-help activity) – the user will need to use the terminal to dig into the datastore and delete the cookie jar manually for that.

Paul: I actually suspect Browse is having a little trouble remembering to log people in, even with cookies, after the XO/Sugar restarts. You may want to check 😀

Me: That leaves us the case when user logs himself out. Now, since the user has done so of his own will, I think that it’d be positively odd to just log him back in automatically the next time he opens social-help. So, you see my dilemma. I think that we should allow the user the choice to log out if he wants to do so. Your suggestions on the issue are welcome, of course.

Paul: In my opinion, clear logging in and logging out options are a necessity if we’re not using on-the-fly anonymous accounts, and users want to track questions they’ve asked/answers they’ve given in the past, etc. The user should definitely be allowed to log out (I’m lending the XO to a friend! I’m changing classroom! Leaving the school!) – and it should try to remember when someone is logged in so they don’t have to do it unnecessarily, but it should state their name somewhere so they can see who they are logged in as (Welcome back Prasoon! Is this you?)

So, I think that I’ll just leave it at that. We may decide to add a Gmail like notice popup for that last part. I’ve asked Paul’s opinion on this in a mail recently.

Anyway, then comes the working with the CSS. I’ve asked Paul’s opinion on this too. He has, just today, let me know of a this guide to aid me in the visual styling. Until now, I have been just fiddling about with the default Discourse styling. Nothing solid though. This guide, I hope, will be of much help. There is another thing – the Discourse custom styling system is a bit lacking. The original styles will all remain and our new style will be applied on top of the existing one. Those acquainted with CSS would know this for quite a problem. We’ll have to pile up !importants for this to work (though theoretically we shouldn’t need those, but that’s just how this works). However, this seems to be the way ahead for now – the Ubuntu team’s discourse has indeed done just this. And so will we, until at least the Discourse team provides users with a better way.

On a side note, I also tried to make a bit of a logo. Come, have a good laugh. My friends did 🙂

Oh well. I suppose we can make something like this.

This week, I’ll be writing quite a bit of CSS, I expect, and giving myself quite a few headaches when the styles don’t work as they should. It always happens when I write CSS. Oh well. See you next week.

Social Help for Sugar: Week 1

Hi all. This summer, I’m doing a GSoC project for sugarlabs. It’s called ‘Social Help for Sugar’. Essentially, I’ll be integrating the Discourse discussion platform with sugar. More details can be found in my application.

Anyway, the first week is past and I’ve been doing some work to get the project started. The first thing I did was to set up discourse locally. This was somewhat more difficult that I assumed it to be. The Discourse team doesn’t provide an official setup guide for local installation. Indeed, the only guides provided are for full-scale production application. And so, I was left with some outdated guides scattered all over the internet. I tried two of those, both resulted in failure. Then, after some desperate searching, I found this guide: https://meta.discourse.org/t/beginners-guide-to-install-discourse-on-ubuntu-for-development/14727 and voila! Discourse was working. I never thought that deploying a simple rails app can be such a pain – the ruby ecosystem is confusing to say the least. Anyway, I’ve enabled admin as well however the automatic mailer isn’t working – smpt ports are blocked in my institute. I’ll find a way around it.

The next thing was make a standalone browser-like activity which will be used to browse the social help forum. This aspect of the project was discussed on the sugar mailing list when I was in the process of making my application. It was decided that having a separate activity was better than relying on the default browser. And so, I cloned the browse activity, made some small changes to it and now, we have a social-help activity.

The next part was to enable a global shortcut for the activity. I had to dig through the code to understand how to do this. This was done as well.

Then, I needed a method using which we could tell the social-help activity which url to launch when it opens. This, I had no idea how to do. So, I went to the IRC but couldn’t get a solution. So, I decided to make a DBus interface that would have a method to open a URL. So, I read a few tutorials on DBus and was going to start tinkering with it when, while taking another look at the code, I found that  there’s a parameter passed to the activity on its startup – a parameter that carries in it a URI to be opened! And, after feeling a little silly, I finally had a solution to launching the social-help activity to any given URI.

The next thing to do will be to create a bunch of topics regarding every major activity and to create a map between activities and discourse topics so that social-help can be launched to the appropriate discussion topic. Then, we’ll take a look at the problem of automatic log-in.