This is how we test smartphones at NextPit

At the outset, we wanted to tell you that we take several measures to ensure objective comparability of the test results. And for the same, we have developed a standardized test procedure. Yes, color space measurement diagrams and line pairs per millimeter graphs are fun to look at. But while we believe that lab tests and scientific tests play a role in some applications, we also take a lot of time to explain (and show) the differences between different devices using real-world examples.

To cite an example, we'll describe how our favorite songs sound differently when using various headphones instead of showing you a frequency response curve. And we'll make sure to show you dozens of different sample photos and numerous photo comparisons between various smartphones so that you can see the results and make your own informed decisions.

Our editorial team has used and tested hundreds of different smartphones, headphones, and wearables over the years. Rest assured, with their experience, NextPit is pretty sure you'll certainly end up getting yourselves the best product based on your requirements.

How do you get the devices you test?

Most of the equipment that is tested is made available to the NextPit editorial team on loan and free of charge. In some cases, it also happens that we also buy the devices. Whatever the case, the origin of the product does not influence our final evaluation. The result is always independent and objective. 

Some manufacturers are also advertising partners at NextPit. However, advertising campaigns or paid articles are clearly marked as such. In addition, we do use also affiliate links and might get some commission for sales. However, this fact will never influence our ratings in any manner.

Our evaluation scheme and the resulting test score is based on the following test criteria, the composition of which will be adapted depending on the device category that is reviewed:

Design & Build Quality

Oh yes, design. That’s something that you can fight over for hours, and it surely lies in the eye of the beholder. But yes, there are some rather objective criteria. A badly balanced phone with cheap materials and weird color combinations will have lower scores than a cleanly designed device made from higher quality materials like glass and metal. To make sure the original reviewer gave the correct score, the grade is also checked by a second editor.

The build quality, on the other hand, is easier to judge. How does the device feel? Does it bend and twist? How well are the buttons and ports made? Is the device scratch-, shock- and/or waterproof? All these factors do play a role in how we reach a final score.

Display

Every device that has a display needs to go through some visual tests. We check display brightness, contrast, refresh rate, and color space, and possible restrictions due to rounded corners and edges or cutouts for the camera or speakers. 

In our reviews, we use two software; HCFR and a Spyder4Elite, to measure contrast, peak brightness, and color reproduction. Furthermore, we rely on the experience and judgment of our editors to review the device. Last but not least, we check the responsiveness of the touchscreen’s digitizer under various conditions, from everyday use to gaming.

Software & Operation

The role that good software plays on a modern gadget is crucial. No matter how great the hardware is, user experience can be marred by poor software. The NextPit editorial team understands this and closely looks at the installed operating system and the changes a manufacturer may have implemented compared to the vanilla version (especially Android).

Another important aspect is the ecosystem built around the operating system regarding app availability, compatible devices, and accessories. Lastly, the update strategy of the manufacturer plays an important role in our review, as it influences the longevity of the device.

Performance

We check both the technical equipment of the device and its performance under load. For that, we run a predefined array of benchmark tests on the device that may vary depending on the device category. 

However, the NextPit editorial team also subjects the device to various everyday tests and closely monitors how it behaves in everyday life. 

Besides the system performance, the wireless connections (LTE / 4G / 5G) are also evaluated and rewarded. Extra points are awarded if the smartphone supports a new wireless standard, such as Wi-Fi 6 or UWB in 2021.

Audio

Our audio tests vary largely depending on the device category. For smartphones, the call quality in both directions (listening and speaking) is important - both with and without ambient noise. Since making calls is still the key feature of a phone (is it?), it is given special attention. For smartphones, we also check the quality of the speaker(s) in terms of bass, clarity, and loudness.

Devices like headphones and speakers – where the audio quality plays an even bigger role are tested using a wide array of songs across various musical genres. For headphones, we also closely check how good/bad the Active Noise Cancellation feature is, if available.

Camera

As smartphones have essentially replaced standalone point-and-shoot cameras,  their camera systems play an important role in our reviews. Among the things we give weightage to in our smartphone camera reviews include; How flexible is the camera system from the ultrawide angle to the telezoom? Does the camera have any extras in terms of hardware and software? Do all the different cameras work together well and is the app user-friendly?

We also examine the image quality under different lighting conditions, from daylight to darkness. How the color reproduction, image noise, and detail reproduction vary under different conditions? Is the camera’s autofocus quick and reliable? And, of course, the selfie camera is important, too.

Last but not least, we take a detailed look at the video quality. We look at the maximum supported resolutions and frame rates while also checking how good/bad the image stabilization feature is. And of course, we also examine how the video quality plays out under different lighting conditions.

Battery

Various factors play a role with rechargeable batteries: How much capacity does the battery offer? Can it perhaps be charged wirelessly, and does it support wired/wireless fast charging? Are there sensible power-saving settings? 

The battery life is especially important for the sensible use of pretty much any modern-day gadget (looking at you, Google Glass!). Accordingly, we also rate the battery in today’s overall context at NextPit and significantly deduct points for comparatively short battery life.

Test procedure

The selection of these test criteria reflects the history of NextPit: Formerly AndroidPIT. While smartphones are still our core business, we have also come around to test devices from different categories. For example, we have featured e-bikes, robot vacuum cleaners, and smart home speakers in the recent past. In such an instance,  an evaluation of the (non-existent) camera is, of course, pointless. Therefore, we completely omit individual evaluation points depending on the device class.

Test score

We test our products according to a clear checklist and with standardized procedures. Nevertheless, consumer electronics products are as diverse as their users and target groups. The NextPit editors assign individual scores to the individual points, which are then resolved into the NextPit star system via various weightage. 

To give the NextPit community a clear indication of the overall rating, we have broken down our scores here:

Test Score at NextPit

5 stars:	This product has absolutely no weaknesses and offers a real technological advantage over the competition in at least one major category.
4.5 stars:	A clear buy recommendation: This product has no weaknesses and is above average in every category.
4 stars	This product is better than average overall. There may be minor weaknesses here and there, but they do not detract from the overall performance.
3.5 stars	Average overall: While this product is good in some areas, it also has noticeable technical flaws that affect its function in everyday use.
3 stars	This product offers below-average performance. There are alternatives on the market that are a better choice in every aspect in this price range.
2.5 stars	No longer a clear buy recommendation. This product offers clear weaknesses and cannot fulfill at least one important feature.
2 stars	Technical problems prevent this product from working as promised. A purchase only makes sense in very special cases.
1.5 stars	This product has severe technical problems and cannot fulfill any of its primary functions. We therefore clearly advise against buying it.
1 star	Smells like fraud: The manufacturer of this product is willfully deceiving its customers. The product in no way fulfills the promised range of functions or simply does not work.

Our test scores do not "age.” This means that they always refer to the time of the test and the latest available firmware available for the device at that time. Before we start testing, we always make sure that the latest software is installed. If the devices are pre-production units or are still running pre-release firmware, we point this out in the test.

Questions that may still move you: Can I also write a test for NextPit?

We are always looking for talented testers from the community who occasionally want to test products and share their results with the NextPit community. We value not only a clear technical understanding but also a clear, structured style of writing. We pay a flat rate for each test report. Since the effort for learning and ensuring the quality on our side is initially high, we only want long-term testers.

Please also understand that we assign the devices to the community testers and that we cannot accept requests for the respective devices. 

Please email us via the contact form if you are interested.