At this point you should already have a PubPub community, a series of pages with information, invitations sent and a very clear objective of what your users are reviewing. But what happens when the user's input about reviewing your information begins?
The CONL team created a methodology in order to be able to properly use the comments generated in the different Pubs. In turn, we designed another small guide to understand how to take advantage of documents uploaded to the platform. In general, these two processes are simple and highly replicable.
For comments on documents uploaded by the administrator, the methodology consists of 3 phases: identification, analysis and resolution. These phases are accompanied by a database that can capture and organize comments more efficiently (after discerning their relevance to the process).
There are five possible types of comments that must be defined from the beginning in order to understand what we may face.
Approvals and disapprovals
This is the most versatile of the five types and can be a precursor or closure for any of the other types.
The relevance of this comment rests on whether or not it has substance in terms of disapproval (if it is approving, it is not relevant for analysis).
These types of comments can be left out of the analysis without having to go to the next stage.
Basically, these are comments that propose the change of words, sentences or entire paragraphs.
They refer to questions of style, semantics, etc.
They can also be accompanied by approval, disapproval, or even theoretical arguments.
Arguments focused on theory or reliable sources
These comments are the most complicated since they involve points of view and biases.
However, they are the most important as they can make great contributions to the review if handled properly.
The relevance of these arguments lies in their sources, in the validity of their data and theoretical frameworks, and in how much they invalidate or disapprove postulates of the documents that are immovable (example: they contradict the human rights approach or propose a theory to analyze the social part that is already expired) as well as in the breadth of the reference framework used to define them (for example: it is in ODS, work tables, previous plan and public consultation).
It is very likely that most people will not cite or be explicit about reliable sources of information. You have to be careful with this situation.
Arguments focused on alternative points of view
These arguments are based on personal or religious beliefs, or imply points of view of doubtful origin.
It is very easy to identify religious and personal comments, but those that cannot be identified need to be analyzed in detail.
This typology was not contemplated for the exercise, but it had to be included since many citizens used the platform to express ideas to improve the State. Although this was not the function of the exercise, the comments were taken into account since they could be used for other exercises or as inputs to create projects.
Once each of the possible types of comments is identified, they must be analyzed. For this, it is enough to have a small model that is based on the form and background of the comment. It is at this time that comments containing the five typologies can be identified, since some types may be focused on form, but have a more important background (substance).
It is important to say that all comments have form and substance, but due to the type of exercise that is carried out, the substance may not be identifiable. For this reason, in addition to the fact that a comment can be identified by one of the five previous typologies, it will also fall in one way or another in one of the two proposed categories: form and substance.
Form: Comments without concise arguments, or data, from unidentifiable frames of reference. They are mainly textual suggestions and / or approvals.
Substance: structured comments that seek to analyze the text from other perspectives. They propose alternative frames of reference or give arguments that justify disapproval, or reinforce the text.
Approvals (I agree).
Disapprovals with less than 5 comments (this is entirely discretionary and each practice management team can change this rule).
Textual modifications without background
If the suggestion is quite precise and a change is necessary, it is taken as a comment
If it is just something that takes away the meaning or is grammatically incorrect, it is not taken into account
The recommendation in this sense of language is that, as far as possible, all these comments are reviewed by someone who is an expert in this area and the theoretical part that needs more analysis is left to the experts in each area.
Textual modifications that go against inclusive language or any attempt to make any problem or group invisible, also against human rights..
Religious arguments are automatically disapproved (obviously if the consultation has some characteristic about some religious belief, it will be totally valid).
That they violate the rules of coexistence (more about this in the section of coexistence rules, terms and conditions and privacy notice.)
Arguments based on personal experiences can serve as testimonials to see if a topic was missing. However, it must be understood that this topic is very likely already in the document or is something outside the scope and responsibilities of the institution that prepares the consultation, in which case it is disapproved.
Very well crafted arguments without sources. They are likely to have no sources due to the nature of the process. However, it is necessary to analyze them to see what they imply.
Everything that was a proposal or project idea was reflected in the database. It is always important to see how much they are relevant to the exercise. For example, there were certain ideas and proposals that were within the scope of the federal executive or the municipalities. This helps decision makers to understand what problems people face, but at the same time, it shows us that not everything can be used due to the responsabilities of the institutions.
Once the comments have been identified and analyzed, a verdict is given as a conclusion of what will happen with that comment. This verdict should be at the same time as the comment capture is taking place. This is only for a matter of time, since capturing all the comments in a database and then analyzing them is a huge time consuming and in this type of process, time is crucial.
Since PubPub does not allow automatic downloading of comments, each approved comment will be transferred to a database. This will help us to have analytics of what types and categories of comments are put, as well as to have control between chapters.
The comment is placed in the database with its respective typology;
And the username.
Done… the proposed change occurs in the review process.
Comments from the same discussion can be stacked.
Note: we recommend using the "blog" comment display functionality found under each Pub. You can also see the comments in each of the comment bubbles on the right side of the Pub, but this becomes very complicated when you have many contributions.
An example of a database can be consulted at the following link: https://participa.conl.mx/pub/5c33k03d/release/1
In this example of a database you can see two important things:
There is a sheet that helps the comment capturer to understand what the typologies are. This tab is for guidance only and has no other function.
The other sheets represent each of the different Pubs to be analyzed.
Example 1 is the most traditional of the Pubs analysis capabilities.
This type of sheet is recommended when there are few comments.
Example 2 is technically the same as Example 2, but a column named “code” was added that uses a semaphore to identify certain types of comments.
This was decided during the analysis and especially for this Pub as this example had an excessive amount of comments.
The traffic light denotes the priority to integrate and use the contributions as inputs.
Red denotes that the comment is not relevant enough to be integrated at this time, but will be considered later.
Yellow denotes comments that are important, but due to lack of time cannot be integrated at this time.
Green denotes inputs that are important and that are so relevant to the process that they should be used as input.
Let us remember that this database example is only a possibility and that each administrator of the query process can create new models and processes. This example worked for the Consejo Nuevo León and should not be understood as a rule for other types of processes. You can use colors too!