Use template.querySelector with variable for selector?

This short answer is, yes. This should work.

I've got this working in a LWC playground at the moment with the following.

My CSS looks like this (I took a guess, as there wasn't anything to go on here):

.contentHidden {
    display: none;
}

My HTML template looks like this:

<template>
    <div data-id="A">This is the div</div>
    <lightning-button onclick={handleClickHide} label="Hide"></lightning-button>
    <lightning-button onclick={handleClickShow} label="Show"></lightning-button>
</template>

The js class looks like this:

import { LightningElement} from 'lwc';

export default class App extends LightningElement {
    handleClickHide(){
        const selector = 'A';

        const theDiv = this.template.querySelector('[data-id="' +selector+ '"]');
        theDiv.className = 'contentHidden';
    }
    handleClickShow(){
        const selector = 'A';

        const theDiv = this.template.querySelector('[data-id="' +selector+ '"]');
        theDiv.className = '';
    }
}

This kind of proves out the mechanics of a dynamic selector, if that's in fact what you're looking to test out.

On the other hand, if you simply need to make items appear/disappear from the DOM, the idiomatic way to do this in LWC is normally to use one of the LWC directives if:true or if:false.

In that case, here's some markeup for that:

<div if:true={show}>Toggle div</div>
<lightning-button label="Toggle" onclick={handleToggle}></lightning-button>

And this is your the JS that you'd need:

export default class App extends LightningElement {

    show = true;

    handleToggle(){
        this.show = !this.show;
    }
}

You can actually see a slightly different example of this in the Salesforce LWC Recipes sample app's implementation of conditional rendering in a LWC component.