Entrar | ¿No tienes cuenta?? Registrarse!
thumb

Test... 1... 2... 3... Test!

Escrito por CipSoft on Wednesday 19 August 2015
the testers...

...almost every Tibian who has ever participated in a private test server or joined thecelebrations of Tibia's 15th anniversary, has seen them running around ingame.
However, who are the people behind the noble pixel figures who are driving around on their uniwheels and are riding on the back of a lady bug?
In preparation of this featured article, we forwarded some questions to the test team and would like to present you the answers we have received from them. Here we go:

CMs: How many members does the test team currently have?

Testers: Currently, the team consists of one leader (Delany) and three further members (Siramal, Seyva, and Bolfrim). While Seyva is the newest tester, Bolfirm works exclusively for CipSoft's innovations department at the moment.

Please note:
As you know, some things have changed in the CipSoft office meanwhile. Delany will take over the lead product manager position within the course of the next months. Since Delany was the lead of the test team we had to search for a new lead there. Once Delany starts his work as lead product manager, Bolfrim will become the new lead of the test team.

CMs: Are there special responsibilities or special tasks for the individual team members? delany

Testers: There are no tasks only one member of the team is responsible for. In general, all testers are capable of working on every task, but, of course, there are specialisations among the team members.

CMs: What are you testing?

Testers: Basically, everything that is developed by CipSoft: a component of Tibia or another product, or tools that are needed for our internal work. We are an independent department inside of CipSoft, so we actually need the assignment from another department to test a feature or new content.

CMs: Why are only technical components being tested but not, for example, if a new quest actually is fun to play?

Testers: Our testing is like the classical software testing. Usability and fun cannot be tested properly by us. That is very often a subjective question of taste and interests. Furthermore, our team is just too small to run any long-time tests concerning fun. Of course, we give feedback concerning usability and fun too, if we notice anything that seems questionable or problematic (in a non-technical way) during our tests. But that is not our focus, it is only an indirect test of those things. Our main task is to test if all technical requirements are fulfilled and if the features are working as intended.

CMs: How much of your daily work do you spend on Tibia?

Testers: Usually, we spend about two thirds of our daily work on Tibia. Though, if it is needed, during peak times, we spend almost 100 % of our daily work on Tibia.

chart2



CMs: Idea - development - test - release. How does it work?

Testers: The product management collects data and drafts a concept of a new feature. After that, the developers and programmers start their work. Once they are done, the feature is tested by the test team. If all technical requirements are fulfilled and the feature works as intended, it is ready to be released.
If there is still some time left, we also check usability, gameplay, spelling, performance, etc. ...
Not every single feature has a predetermined workflow regarding development and release. Some things with a lower priority are done once there is time.

CMs: How do you receive bug reports?

Testers: Ingame reports from tutors are automatically forwarded to our internal tool where we check and verify them. Other reports (via email, for example) are processed by our customer support team and then forwarded to us as well.

CMs: How do you prioritise the fixing of bugs?

Testers: Basically, there are two factors that are important for a prioritisation. The first one is the severity of a problem. That means how much damage a bug can potentially cause. The severity is estimated by the test team, sometimes in cooperation with the support team or the community management. On the basis of this estimation the product management then decides the second factor, the priority itself. The priority determines how urgent a fix is needed compared to other fixes. Although, the severity has influence on the decision of the product managers, it is up to the priority ranking how fast a bug will be addressed and a fix will be released.

bolfrimCMs: After every update, some players keep saying that there are more and more bugs being released with the actual update compared to the past. Is that actually true?

Testers: According to our internal statistics, we cannot confirm this. We compare the total amount of bugs we find internally before a release with the total amount of bugs which are found after an update has been released. This ratio has been improving ever since the first tester has joined the team. Nevertheless, due to Tibia becoming more complex through the years, the absolute number of bugs has indeed increased. Also, that number depends on the amount of changes that are part of an update. For example, if we add three new cities in one update, the number of bugs is likely to be three times higher than if only one city is added. The complexity and the size of changes has increased massively during the years. That might explain the perception you have mentioned.

CMs: During the last updates, less and less private testers have been online. Has this influenced the quality of bug reports? If so, are there any plans to increase the numbers of private testers in the future?

Testers: The fact that there have been fewer testers did not have any impact on the quality of the reports but on the quantity. Furthermore, fewer testers also means that there are most likely some functions and areas noone has found or visited and therefore they were not tested.
We have already invited more players to the private test phase for this year's summer update and we will of course evaluate the whole test and then make further plans for the next update.

CMs:Do you play Tibia? If so, what world types do you play on and what are your levels?

Testers: Each one of us has a private Tibia account, but the intensity and the frequency of our playing sessions vary.
Siramal, for example, started playing Tibia very actively when he joined CipSoft. He plays on an Optional PvP game world and still logs in to play some up to today, especially to participate in events or to trade with others. He also has another character on an Open PvP game world, but he prefers a peaceful environment.
In general, if a new colleague joins he or she usually gets a lot of time to get in touch with Tibia and to get to know the game better.

CMs: How is your everyday work in the office?

Testers: People, who do not know much about the games industry, usually say: "Oh, so you are playing games the whole day!" Well, this is definitely not true.
We start the day with checking our emails and our internal bug tracker. Then we verify new reports. If we can reproduce them, the product manager will prioritise them and forward them to the right people. Of course, fixes are tested as well. There are simply a lot of other things to be tested as well, and that makes a huge part of our daily work.

In our office, we have a board which gives us an overview about our current tasks and their progress. We update the board once per day. If we have finished one task, we can always take a new one from there. Anyway, most of the time we are working on several tasks at the same time.

Furthermore, our phones ring quite often. We are involved in almost all new features, content projects, etc. We testers have a good overall view. So, we are a good contact point for any queries from other departments. Sometimes, it is better to resolve questions personally; therefore colleagues from other teams visit us quite often.

We usually spend our breaks in our break room where we eat, drink and talk with other colleagues. Sometimes, we also spend some time in our recreational room and play some games there.

CMs: Are there "rush hours" which influence your daily work and your way of working?

Testers: Yes, of course, there are rush hours. It can happen that we have to drop everything else and focus on an urgent problem in Tibia. However, that does not influence our way of working. We have to remain level-headed in stressful situations. Often, it is the test team which advises to reschedule a release to prevent loss of quality. Sometimes, we also have to put in extra hours but that is not a big problem since we have the opportunity to take time off when there is less to do.

Testers: Are there any major test projects currently or what was your last major project?

Testers: There are major projects frequently, such as the summer and winter updates, for example. Past major projects have been the new payment system, the Premium Scrolls, and the Tibia Flash client for example. Anyway, major test projects might differ from tester to tester.

Seyva:
"My last major project was to test the new guild features, and I really had fun testing the whole thing. Right now, we are testing the summer update and the new ingame store. This is probably the biggest project I have been testing for Tibia yet. It is indeed super interesting, because it is very complex and a lot of components are affected."

CMs: How do you become a tester and what skills are needed? sevya

Testers: There is not really a professional education for testers. The skills you need depend on the sector you are working in. As a game tester, you should be well organised and stress-resistant, work independently, do further studies and you should have an unerring eye for systems and "breakpoints"; because what do testers do? We basically try to break a system. We try to find bugs and problems within a feature by all available means, even if the developer created a feature or content with a lot of love and a lot of work. Therefore, you also need to be tactful when you bring the bad news that the feature is not working as intended. Furthermore, some basic knowledge of game design, UI, and scripting are of advantage. Good English skills are a must, and you should have a certain degree of curiosity and determination.

CMs: Did it happen that you were angry with yourself because you did not find a major bug during your testing phase?

Testers: That happens. It is our job to find bugs before they are implemented into the game. Every bug we find is like a small achievement for us. If then, after all the testing, a bug still makes it into the game it is, of course, very annoying to us. To be honest, though, we will never be able to guarantee a 100% bug free release. Tibia is just too big and too complex and our time is too limited to make such a promise.
Therefore, it is important to consider whether or not it would have been possible to find a bug with reasonable effort. Every time we overlook a bug, we question ourselves and analyse, if there are ways to improve or if a change of our testing methods is needed in order to find such bugs easier in the future. Quite often we have to answer this question with a clear no. Nevertheless, overlooking a major bug feels indeed like a personal defeat sometimes.

siramalSiramal:
"Some years ago, we added a new big dungeon as part of an update. We tested the area as always. Map bugs and creature bugs were forwarded and fixed. Even during the external test everything worked as intended. The surprise was not long in coming. After a few days, we noticed that a whole level of the dungeon was inaccessible. Stairs or a rope spot were simply missing... and of all things, parts of a taming item were there... Well, learning never stops and when testing an area today we do not just jump there anymore by using a god spell, but test in advance, if the new area is accessible at all."

Seyva:
"I still remember the bug when the NPCs had stopped yelling/barking after the release of the new guild system last autumn. I'd never dreamt that redoing the guild chat could break something concerning NPCs."

Delany:
"It is often very annoying if we try to reproduce bug reports, but just cannot find the bug even if we do everything right according to the report. Usually, a very small detail is missing in the report and therefore in the testing scenario as well. If after long research, the programmer finds the bug within the code, we see what was missing. For example, we just tested at the wrong place. Something like that can be very frustrating."

CMs: Thank you for your time, testers!

Tibians, we hope you enjoyed the answers from the testers and that we were able to give you an overview about their work.

See you in Tibia!
Your Community Managers!