Maemo.org team/Creating an election

These are the steps required to create a new election for the Community Council - essentially the same process applies for a referendum.


Contents

[edit] The new Admin Interface

The new admin interface can be accessed by active system administrators and Council. The interface was designed to handle simple to medium election setups. Very complex elections are possible via the interface, but may at times be easier to do with direct SQL access.

Admin Control Panel
The admin control panel is the primary page seen after logging in. From here, you select the Election you wish to work on, and the action you wish to do involving this election. Once both drop downs are properly updated, hit the "Apply" button to move forward in your task. You can always return to this page (without saving changes) by hitting the "Return to ACP" button.

[edit] Vital steps in starting a new election

Starting a new election requires a few updates to occur in the voting both:

Updating the electorate
The electorate is the table of valid voters. This still under construction at the time of this writing, and is a highly manual process still. To update this, an admin must copy columns of the login table from the Midgard system into a table in the voting database. (Admin Note: The script "update-midgard-table.php" currently does this, but must be run as root to accomplish this task.) From there, the interface can be used to update the e-mail and/or find and add new members to the electorate table.
Create New Election
The second step is setting up the actual election/referendum. This sets up a new election in the database, and provides selections for settings for the election. Note: Always verify the dates of the election, and ensure they are in the future. The admin interface will lock out editing of an election if it's slated to start on or before the date indicated. Once the election is added, it should auto-forward you to the next step in the process.
Edit Options/Candidates
Here, the Apply button will loop you back to this page so you can enter multiple selections at once. You can return here from the main menu to make changes until the day of the election. When all candidates are entered, select "Next" to move to the next step.
Make/Edit Mail Template
The next step is creating a mailing template for this election. This sets up the e-mail form used to send tokens to voters. A default template will be loaded based on the type of election, or from a previously edited and saved template, if you return to this step later. Note: For elections where you will not be sending an e-mail (eg cloned token elections), this stop is optional. After saving the template, you will be forwarded on to the token interface.
Make/Clone/Mail Tokens
This interface changes based on the state of the token database. If tokens already exist for this election, an indicator will show you how many tokens exist. This can be compared to the number of valid electorate to be sure all members have tokens. Here, you can make new tokens for an election or clone them from another election. Once tokens are present options to mail out tokens will show up, and options to clone or make new tokens will go away. Once tokens are made and mailed out, or cloned, you can return to the control panel. Your election should start on the dates given, and people should receive the tokens over the next few days.

[edit] A few notes on token and token generation

Tokens are random bits made to allow anonymous voting access. They are destroyed as the voter "pulls the lever" and makes their choices permanent in the voting interface. For this reason, one can not and should not clone tokens from an active or past election (and the admin interface will not allow such actions to happen).
Tokens also are electorate specific, so updating the user list and making new tokens just for those users is somewhat of a task. It can be done, and hopefully soon the interface will handle that task. But currently it's a manual process to do so. (See notes below on how to do this.) With the addition of the admin interface, the hope is to allow future elections to all use the same electorate group, by appending new members and updating old ones. Elections prior to this reset the electorate interface every time.

[edit] Other Admin Functions

Below is a list of all admin functions and how they currently work.

Add An Admin Function (Election Selection Irrelevant)
This allows existing admins to add new admin accounts. The rights on sub-admins are the same as the admin creating them. Note that the username and e-mail must be double checked, and that the initial login password is mailed directly to the user. This should be used by out-going Council to add new Council to the admin interface.
Remove An Admin Function (Election Selection Irrelevant)
For super admins, this allows the addition or removal of any account. For standard admins, this allows them to remove themselves as an admin. This functionality is meant mainly as a maintenance function, and as a way for outgoing Council to relinquish admin rights as they exit.
Change User Info (Election Selection Irrelevant)
This option allows you to update your own user information, be that your e-mail address or password for the system. Super admins may be able to update e-mail and/or force a new user password/token to be sent to other users.
Create New Election (Election Selection Irrelevant)
This one is somewhat self-explanatory. It creates a new election in the election database. That user is walked through all steps of creating an election from this menu item.
Edit Existing Election (Valid on future elections only)
This item allows you to edit existing future elections. It does not allow users to edit active or past elections. For this reason, when creating or editing an election, be conscious of the date selections, as after you hit save, that start date is when it becomes non-editable.
Delete Existing Election (Valid on future elections only)
This allows you to remove a pending election, in case you screwed up in some way that would make editing it not an option. Use this sparingly, as most things can be edited to be fixed.
Edit Options/Candidates (Valid on future elections only)
This allows you to update, add, or remove candidates or choices from elections and referendums. You can change these in the same way you would edit an election, and changes are bared once the election is active (or past).
Make/Edit Mail Template (Valid on future elections only)
This creates or edits the template used to mail out information about the election selected. This is used by the e-mailer to send out tokens for this election. For cloned elections, this step can be skipped.
Make/Clone Mail Tokens (Valid on future elections only)
This interface is somewhat complex. It allows admins to create new tokens for new elections. It also allows for cloning of other future elections, so if you want to hold two or three elections with the same token for each user, that's viable. (This is done to cut down the number of e-mail sent out, to keep us off spam lists.) This interface changes based on if tokens are available, if other future elections are available for cloning, etc.
Compute Results of Vote (Valid on past elections only)
This function allows for the entering of election results. Be default, the system should auto-generate the results and present them for entering into the database as final an approved. This should work for referendums and runoff elections now. If an error occurs, consult and admin.


[edit] Resending Vote Tokens

Under the new interface, any voter can force their own vote token to be mailed to them. The admin may need to update the user's e-mail address with the web interface as documented above, but should never have to extract, e-mail, or see a user token with this system. The voter simply needs to visit the login page for the on-going election or referendum they want to participate in, and use the "Get Token" button. This will ask them for their username and e-mail address. If they match (in the voting database) and the voter has a valid token, it will re-mail them automatically.


[edit] This is how Elections were done before the admin interface

Get list of maemo.org accounts
Request a dump of maemo.org accounts including karma and account creation date from Henri or Neithan at the karma cut-off date. The format for each row is: 
username,"Real name",firstname,surname,emails(; separated),"join date",karma,talk profile URL
Create database if necessary
The schema is: 
DROP TABLE IF EXISTS `electorate`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `electorate` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`username` varchar(20) DEFAULT NULL,
`firstname` varchar(50) NOT NULL DEFAULT '',
`lastname` varchar(50) NOT NULL DEFAULT '',
`email` varchar(50) NOT NULL DEFAULT '',
`karma` int(11) NOT NULL DEFAULT '0',
`creation_date` datetime DEFAULT NULL,
`talk_id` varchar(70) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=32822 DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;
Create election in the database
For the Q3 2010 election, that means running the following on the maemoelections database on test.maemo.org: 
/* Fields are ID, type ('elections' or 'referendum'), name, start date, end date, # of seats, question */
insert into elections values(10, 'elections', 'Maemo Community Council elections, Q3 2010', '2010-09-16 00:00:00', '2010-09-22 23:59:59', 5, 'In order of preference, which of the following candidates do you want to represent you on the Maemo Community Council?');


/* Fields are id, election id, name - id is auto-increment */
INSERT INTO election_choices VALUES
  (0, 10, 'Andrea Grandi (andy80)'),
  (0, 10, 'Tim Samoff (timsamoff)'),
  (0, 10, 'Rüdiger Schiller (chemist)'),
  (0, 10, 'Cosimo Kroll (zehjotkah)'),
  (0, 10, 'Andrew Flegg (Jaffa)'),
  (0, 10, 'Stephen Gadsby (sjgadsby)'),
  (0, 10, 'Felipe Crochik (fcrochik)'),
  (0, 10, 'Robin Burchell (w00t)'),
  (0, 10, 'Kathy Smith (revdkathy)'),
  (0, 10, 'Attila Csipa (achipa)')
;
Insert electorate into database
Insert the rows into your database using the following (note, we ignore the talk profile URL now, but there is still a column for it in the database): 
/* Truncate old electorate table and old temp tokens */
TRUNCATE electorate;
TRUNCATE election_tmp_tokens;
 
/* Load data file using LOAD DATA INFILE (or mysqlimport) */
LOAD DATA LOCAL INFILE '/path/to/data/users.csv' INTO TABLE electorate
   FIELDS terminated BY ',' OPTIONALLY ENCLOSED BY '"'
   (username,@dummy,firstname,lastname,@email,creation_date,karma,@talk)
   SET email=substring_index(@email,';',1), 
       talk_id=substring_index(@talk,'=',-1)
;
/* This is a complicated line, worth explaining.
 * LOAD DATA INFILE basically does a series of inserts, one for each 
 * line of data in the input file.
 * The part in brackets matches fields or variables to entries in the 
 * data line, with separator, quote and line ending bevahiour as specified.
 * We drop the "Real name" field. The email field contains semi-colon
 * separated list of all the email addresses associated with the account,
 * so we search the field for a semicolon, and just take the first one.
 * The talk field contains a URL with a talk_id after "=", which is what we
 *  want, so we look for = and take everything after the first occurrence.
 */
 
/* Delete from electorate anyone with insufficient karma */
DELETE FROM electorate WHERE karma < 10;
/* Delete accounts created too recently (3 months before end of election) */
DELETE FROM electorate WHERE creation_date > '2010-06-23';

It is also possible to filter on older talk_ids, even if the maemo account was created recently. Ask Reggie for the cut-off Talk user id for the electorate date, and include the cut-off in the "delete from" for recently created accounts.

Extract eligible candidates

Generating the list: 

 mysql -e 'select username, firstname, lastname, karma  from electorate where karma>100 order by lastname'> eligible.txt

Awk script to generate the table: 

BEGIN{ FS="\t";
print "{|\n|-\n! Username\n! Real name\n! Karma";
}
 
{
print "|-";
username=$1;
print "| [http://maemo.org/profile/view/" username ".html " username "]";
print "| " $2 " " $3;
print "| " $4;
}
END{
print "|}";
}
Generate election tokens and send out emails
Tokens and emails are created with scripts in the maemo2midgard repository - create-tmp-tokens.pl and mail-instructions.pl Database credentials are in a private local file called config.pl
We need to modify the email instructions (instructions.txt) every election with the candidate list, links to election materials, etc. First we generate the tmp tokens, insert them in the database by running the resulting file through the mysql client, and then using the instructions and the generated maildata.txt file as inputs, we mail the instructions out. 
./create-tmp-tokens.pl <election_id> <tokens_file> <maildata_file>
mysql -u<dbuser> -p<dbpasswd> -h<dbhost> < <tokens_file>
./mail-instructions.pl <maildata_file> <instructions>

And we're all done! The election will run until its closing date, at which point you can generate the results

Resending tokens

Sometimes token emails get caught in spam filters, or accidentally deleted, or the first email address associated with their account is dead, or whatever. You can find the anonymous token associated with someone as long as they have not voted yet.

In a Plain Old mysql client, run the following query: 

 SELECT e.id, e.username, e.firstname, e.lastname, e.email,
        t.election_id, t.tmp_token
 FROM electorate AS e, election_tmp_token AS t
 WHERE e.id=t.member_id AND e.username='<username>';

You can also look using "LIKE" on the username, first name or last name if you need to. This will give you the email/tmp_token pairing which the person can then use to vote, and which you can then send on to them, after verifying that they are who they say they are.

Retrieved from "http://wiki.maemo.org/Maemo.org_team/Creating_an_election"