TDD with large data in Python

Question

I wonder if TDD could help my programming. However, I cannot use it simply as most of my functions take large network objects (many nodes and links) and do operations on them. Or I even read SQL tables. Most of them time it's not really the logic that breaks (i.e. not semantic bugs), but rather some functions calls after refactoring :)

Do you think I can use TDD with such kind of data? What do you suggest for that? (mock frameworks etc?) Would I somehow take real data, process it with a function, validate the output, save input/output states to some kind of mock object, and then write a test on it? I mean just in case I cannot provide hand made input data.

I haven't started TDD yet, so references are welcome :)

See this similar (but not identical) question: stackoverflow.com/questions/842476/… — Raedwald
– Raedwald, Commented Mar 11, 2012 at 17:03

Jon Cairns · Accepted Answer · 2012-03-11 15:09:35Z

3

You've pretty much got it. Database testing is done by starting with a clean, up-to-date schema and adding a small amount of known, fixed data into the database. You can then do operations on this controlled environment, knowing what results you expect to see.

Working with network objects is a bit more complex, but it normally involves stubbing them (i.e. removing the inner functionality entirely) or mocking them so that a fixed set of known data is returned.

There is always a way to test your code. If it's proving difficult, it's normally the code design that needs some rethinking.

I don't know any Python specific TDD resources, but a great resource on TDD in general is "Test Driven Development: A Practical Guide" by Coad. It uses Java as the language, but the principles are the same.

answered Mar 11, 2012 at 15:09

Jon Cairns

12k4 gold badges44 silver badges67 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Community · Accepted Answer · 2017-05-23 12:21:12Z

most of my functions take large network objects

Without knowing anything about your code, it is hard to assess this claim, but you might want to redesign your code so it is easier to unit test, by decomposing it into smaller methods. Although some high-level methods might deal with those troublesome large objects, perhaps low-level methods do not. You can then unit test those low-level methods, relying on integration tests to test the high-level methods.

Edited:

Before getting to grips with TDD you might want to try just adding some unit tests.

Unit testing is about testing at the fine-grained level: see my answer to the question How do you unit test the real world.
You might have to introduce some indirection into your program, to isolate parts that are impossible to unit test.
You might find it useful to decompose your data into an assembly of smaller classes, which can be tested individually.

Collectives™ on Stack Overflow

TDD with large data in Python

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related