Django REST Framework: TestCase is not returning correct queryset

Question

I have created an API, using DRF, for products in an inventory that can be accessed by the following endpoint url(r'products/$', views.InventoryList.as_view(), name='product-list').

When issuing a GET request via postman, I get the correct queryset back, which is a total of 11 products:

[
    {
        "id": 1,
        "name": "Biscuits",
        "description": "Papadopoulou Biscuits",
        "price": "2.52",
        "comments": [
            {
                "id": 1,
                "title": "First comments for this",
                "comments": "Very tasty",
                "rating": 8,
                "created_by": "xx"
            }
        ]
    },
    {
        "id": 2,
        "name": "Rice",
        "description": "Agrino Rice",
        "price": "3.45",
        "comments": []
    },
    {
        "id": 3,
        "name": "Spaghetti",
        "description": "Barilla",
        "price": "2.10",
        "comments": []
    },
    {
        "id": 4,
        "name": "Canned Tomatoes",
        "description": "Kyknos",
        "price": "3.40",
        "comments": []
    },
    {
        "id": 5,
        "name": "Bacon",
        "description": "Nikas Bacon",
        "price": "2.85",
        "comments": []
    },
    {
        "id": 6,
        "name": "Croissants",
        "description": "Molto",
        "price": "3.50",
        "comments": []
    },
    {
        "id": 7,
        "name": "Beef",
        "description": "Ground",
        "price": "12.50",
        "comments": []
    },
    {
        "id": 8,
        "name": "Flour",
        "description": "Traditional Flour",
        "price": "3.50",
        "comments": []
    },
    {
        "id": 9,
        "name": "Oregano",
        "description": "Traditional oregano",
        "price": "0.70",
        "comments": []
    },
    {
        "id": 10,
        "name": "Tortellini",
        "description": "Authentic tortellini",
        "price": "4.22",
        "comments": []
    },
    {
        "id": 11,
        "name": "Milk",
        "description": "Delta",
        "price": "1.10",
        "comments": []
    }
]

I wrote then a test (using pytest ) to test this endpoint:

import pytest
import pytest_django
from django.urls import reverse
from rest_framework import status
from rest_framework.test import APITestCase

class TestInventoryList(APITestCase):
    @pytest.mark.django_db
    def test_get_product_list(self):
        url = reverse('product-list')
        response = self.client.get(url)
        print(response.json())
        self.assertEqual(response.status_code, status.HTTP_200_OK)
        self.assertEqual(len(response.json()), 11) # <-- TC fails here

but it fails since response.json() returns only the first 9 objects:

[{
    'id': 1,
    'name': 'Biscuits',
    'description': 'Papadopoulou Biscuits',
    'comments': [],
    'price': '2.52'
}, {
    'id': 2,
    'name': 'Rice',
    'description': 'Agrino Rice',
    'comments': [],
    'price': '3.45'
}, {
    'id': 3,
    'name': 'Spaghetti',
    'description': 'Barilla',
    'comments': [],
    'price': '2.10'
}, {
    'id': 4,
    'name': 'Canned Tomatoes',
    'description': 'Kyknos',
    'comments': [],
    'price': '3.40'
}, {
    'id': 5,
    'name': 'Bacon',
    'description': 'Nikas Bacon',
    'comments': [],
    'price': '2.85'
}, {
    'id': 6,
    'name': 'Croissants',
    'description': 'Molto',
    'comments': [],
    'price': '3.50'
}, {
    'id': 7,
    'name': 'Beef',
    'description': 'Ground',
    'comments': [],
    'price': '12.50'
}, {
    'id': 8,
    'name': 'Flour',
    'description': 'Traditional Flour',
    'comments': [],
    'price': '3.50'
}, {
    'id': 9,
    'name': 'Oregano',
    'description': 'Traditional oregano',
    'comments': [],
    'price': '0.70'
}]

A couple of observations here:

The queryset returned in my test case does not contain the comments for my first product even-though when accessed via postman I can see the comments. Comments is a different django model which is accessed through this nested endpoint: url(r'^products/(?P<product_id>[0-9]+)/comments/$', views.CommentsList.as_view())
I inserted the last two products as well as the comment for my first product (none of which is returned in the latter queryset) using POST and an API auth token. Is this an information that I should somehow include in my test case?

EDIT

models.py

    from django.db import models
from django.contrib.auth.models import User

class Product(models.Model):
    name = models.CharField(max_length=255)
    description = models.TextField()
    price = models.DecimalField(decimal_places=2, max_digits=20)


class Comments(models.Model):
    product = models.ForeignKey(Product, related_name='comments')
    title = models.CharField(max_length=255)
    comments = models.TextField()
    rating = models.IntegerField()
    created_by = models.ForeignKey(User)

urls.py

from django.conf.urls import url
from . import views

urlpatterns = [
    url(r'products/$', views.InventoryList.as_view(), name='product-list'),
    url(r'^products/(?P<product_id>[0-9]+)/$', views.InventoryDetail.as_view()),
    url(r'^products/(?P<product_id>[0-9]+)/comments/$', views.CommentsList.as_view()),
    url(r'^products/(?P<product_id>[0-9]+)/comments/(?P<comment_id>[0-9]+)/$', views.CommentsDetail.as_view()),
]

views.py

from rest_framework import generics
from rest_framework.permissions import IsAuthenticatedOrReadOnly
from .models import Product, Comments
from .serializers import ProductSerializer, CommentSerializer
from .permissions import IsAdminOrReadOnly, IsOwnerOrReadOnly


class InventoryList(generics.ListCreateAPIView):
    queryset = Product.objects.all()
    serializer_class = ProductSerializer
    permission_classes = (IsAdminOrReadOnly, )
    lookup_url_kwarg = 'product_id'


class InventoryDetail(generics.RetrieveUpdateAPIView):
    queryset = Product.objects.all()
    serializer_class = ProductSerializer
    permission_classes = (IsAdminOrReadOnly, )
    lookup_url_kwarg = 'product_id'


class CommentsList(generics.ListCreateAPIView):
    serializer_class = CommentSerializer
    permission_classes = (IsAuthenticatedOrReadOnly, )
    lookup_url_kwarg = 'product_id'

    def perform_create(self, serializer):
        serializer.save(created_by=self.request.user, product_id=self.kwargs['product_id'])

    def get_queryset(self):
        product = self.kwargs['product_id']
        return Comments.objects.filter(product__id=product)


class CommentsDetail(generics.RetrieveUpdateDestroyAPIView):
    serializer_class = CommentSerializer
    permission_classes = (IsAuthenticatedOrReadOnly, IsOwnerOrReadOnly)
    lookup_url_kwarg = 'comment_id'

    def get_queryset(self):
        comment = self.kwargs['comment_id']
        return Comments.objects.filter(id=comment)

permissions.py

from rest_framework.permissions import BasePermission, SAFE_METHODS


class IsAdminOrReadOnly(BasePermission):
    def has_permission(self, request, view):
        if request.method in SAFE_METHODS:
            return True
        else:
            return request.user.is_staff


class IsOwnerOrReadOnly(BasePermission):
    def has_object_permission(self, request, view, obj):
        if request.method in SAFE_METHODS:
            return True

        return obj.created_by == request.user

Things coming to mind: different database or different pagination settings used for testing. — hoefling
– hoefling, Commented Mar 31, 2018 at 19:39
Do you have any specific check in mind that I can do to verify both of your suggestions? As far as I remember, I haven't changed anything regarding pagination and I only use one DB in my virtual env. Do you think the problem may has something to do with the API token? — kingJulian
– kingJulian, Commented Mar 31, 2018 at 19:56
As for the db, just check whether you have the entities in the table: add assert Product.objects.get(pk=10).name == 'Tortellini' to the test, does the test pass the line? — hoefling
– hoefling, Commented Apr 1, 2018 at 10:27
As for pagination, on second thought it's probably not an issue here as you're also missing the comments by the first product. — hoefling
– hoefling, Commented Apr 1, 2018 at 10:30
Why don't you use something like setUpClass or setUpTestData for testing your views? It will be more robust and independent of your production data. — cezar
– cezar, Commented Apr 3, 2018 at 8:59

John Moutafis · Accepted Answer · 2018-04-03 21:30:36Z

4

+25

I suspect, (without having your product model at hand) that you are not getting all the elements from the products table, for the following reasons:

You created your first 9 elements manually, without registering them to a specific user.
Afterward, you added an authentication method (TokenAuthentication) and create some users with access tokens.
Because you added an authentication method, you probably added @permission_classes((IsAuthenticated,)) / permission_classes=(IsAuthenticated,) to your product-list view.
That restricts any unauthenticated user from accessing the product-list.
The unauthenticated-anonymous users will only see the anonymous elements of your database.
You added the next 2 elements and the comment with one of the registered users, which in turn registered those elements to the user-creator, therefore you cannot access them without an authenticated user.

To access resources that need authentication from the DRF's test client, you need to authenticate your user first.
You can use force_authenticate method:

class TestInventoryList(APITestCase):
    def setUp(self):
        self.req_factory = APIRequestFactory()
        self.view =  views.InventoryList.as_view({'get': 'list',})

    @pytest.mark.django_db
    def test_get_product_list(self):
        url = reverse('product-list')
        request = self.client.get(url)
        force_authenticate(request, user=YOUR_USER)
        response = self.view(request)
        print(response.json())
        self.assertEqual(response.status_code, status.HTTP_200_OK)
        self.assertEqual(len(response.json()), 11)

This test assumes that you list method returns Products.objects.all()

As @cezar points out, testing a view against real data is prone to fail (for example, when you add a new element, the self.assertEqual(len(response.json()), 11) will fail)

You should consider mocking your responses to create an isolated environment.

I tend to use a combination of factory_boy and django-nose (pytest works as well).

edited Apr 3, 2018 at 21:30

answered Apr 3, 2018 at 11:16

John Moutafis

23.3k11 gold badges74 silver badges118 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

kingJulian Over a year ago

Hi. Thanks for your reply. First off, I edited my question to add the model. Secondly, your logic is right. I wanted only admin users to add products so I had the POST method allowed only to that type of users. I then created an endpoint (see urls.py) that would return a token to a client after passing user name and password. I then created a django superuser and passed this username and password to get the token back. I then created the appropriate permission classes (see corresponding file) and did a POST request using my token.

kingJulian Over a year ago

With regards to your code; although it was helpful in the sense of pointing me towards the right direction (i.e. force_authentication function) it did not work for me. Specifically it said that there's an argument error in the self.view line. Apart from my code cannot find, for some reason, the user I created above. Thus the force_authenticate(request, user=YOUR_USER) line also fails. This is weird because I can see the user in my auth_user table but when I perform a User.object.all() query I cannot retrieve it.

Collectives™ on Stack Overflow

Django REST Framework: TestCase is not returning correct queryset

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related