How to find and replace a series of repeated words in an array of string?

Question

I have this array of string:

let prompts = [
  "This is some word",
  "This is some more word",
  "This is some more word",
  "This is some more word",
  "This is some more word",
  "This is something else"
];

Is it possible to write a function that can find repetitions in the array (e.g: This is some more word) and replace it with "REPEATED" while keeping the first one intact. We also need the ability to set a number for the words to be considered as "repetitions" otherwise it will match pretty much everything.

When I call the function. the output will be:

[
  "This is some word",
  "This is some more word",
  "REPEATED",
  "REPEATED",
  "REPEATED",
  "This is something else"
];

This has been troubling me for a long time before I reach out for help, because the string in the prompts variable is totally random and I want to eliminate repetitions in OpenAI prompts. Thus this question.

Thanks in advance!

Update: This is the closest I got so far

function detectRepetitions(arr, minWords) {
  const uniqueWords = new Set();
  const repeatedWords = new Set();

  // Iterate over the array and find repeated words
  for (let i = 0; i < arr.length; i++) {
    const words = arr[i].split(" ");

    for (let j = 0; j <= words.length - minWords; j++) {
      const subArray = words.slice(j, j + minWords);
      const subString = subArray.join(" ");

      if (uniqueWords.has(subString)) {
        repeatedWords.add(subString);
      } else {
        uniqueWords.add(subString);
      }
    }
  }

  // Replace repeated words with 'REPEATED'
  const result = arr.map((sentence) => {
    let words = sentence.split(" ");
    let repeatedFound = false;

    for (let i = 0; i <= words.length - minWords; i++) {
      const subArray = words.slice(i, i + minWords);
      const subString = subArray.join(" ");

      if (repeatedWords.has(subString)) {
        if (repeatedFound) {
          words.splice(i, minWords, "REPEATED");
        } else {
          repeatedFound = true;
        }
      }
    }

    return words.join(" ");
  });

  return result;
}

let prompt = [
  "This is some word",
  "This is some more word",
  "This is some more word",
  "This is some more word",
  "This is some more word",
  "This is something else",
];

const minWords = 4;
const result = detectRepetitions(prompt, minWords);
console.log(result);

What effort have you made? Please share that as a text-based minimal reproducible example — evolutionxbox
– evolutionxbox, Commented Jul 7, 2023 at 19:03
It should be replaceable by regex using global replacement. See stackoverflow.com/questions/1162529/javascript-replace-regex — sativay
– sativay, Commented Jul 7, 2023 at 19:11
Ask it to amend the function where it only to replaces the subsequent repeated strings, not the first. — evolutionxbox
– evolutionxbox, Commented Jul 7, 2023 at 19:11

M0nst3R · Accepted Answer · 2023-07-07 19:20:14Z

3

You can simply reduce() the initial array into a new one where you build it item by item, and if an old value is encountered again, you'd fill the new array with 'REPEATED' instead. You can assign this new array to the old one if you want to override it.

let prompts = [
    "This is some word",
    "This is some more word",
    "This is some more word",
    "This is some more word",
    "This is some more word",
    "This is something else"
];
prompts = prompts.reduce((acc, cur) => (
    acc.includes(cur) ?
        [...acc, 'REPEATED']
        :
        [...acc, cur]
), []);
console.log(prompts);

answered Jul 7, 2023 at 19:20

M0nst3R

5,3131 gold badge28 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Fariz Luqman Over a year ago

Thank you! reduce() is the answer! Thanks for the answer :)

Maziar Rumiani · Accepted Answer · 2023-07-07 19:19:46Z

1

You could use some nested loops to achieve this:

let prompts = [
  "This is some word",
  "This is some more word",
  "This is some more word",
  "This is some more word",
  "This is some more word",
  "This is something else"
];

function replaceRepeated(arr) {
  for (let i = 1; i < arr.length; i++) {
    for(let j = 0; j <i; j++){
      if(arr[i] === arr[j]){
        arr[i] = 'repeated'
      }
    }
  }
  return arr;
}
console.log(replaceRepeated(prompts))

Also here's a video I made about for loops in JavaScript: for loops in js

answered Jul 7, 2023 at 19:19

Maziar Rumiani

7511 gold badge4 silver badges12 bronze badges

Comments

ee-4-me · Accepted Answer · 2023-07-07 19:41:27Z

O(n) answer with hash

(Nested loops answer will be O(n^2) time complexity. i.e. poor performance for large array of strings)

This should also be faster than the reduce answer as javascript .includes() performs a linear search on the growing acc, which makes the function O(n^2) time complexity.

let prompts = [
  "This is some word",
  "This is some more word",
  "This is some more word",
  "This is some more word",
  "This is some more word",
  "This is something else"
];

const make_repeated = (prompts) => {
  const hash = {}
  for (let i = 0; i < prompts.length; i++) {
    const sentence = prompts[i];
    if (hash[sentence] !== undefined) prompts[i] = 'REPEATED'; // i.e. seen before
    hash[sentence] = 0; // intialize
  }
  
  return prompts;
}

console.log(make_repeated(prompts));

// ["This is some word", "This is some more word", "REPEATED", "REPEATED", "REPEATED", "This is something else"]

dean glükler · Accepted Answer · 2023-07-07 19:52:28Z

1

const usedPrompts = {}

const promptsCleaned = prompts.map((prompt) =>  {
  if (usedPrompts[prompt])  {
    return 'REPEATED'
  }
  usedPrompts[prompt] = true

  return prompt
})

promptsCleaned will hold your desired array.

answered Jul 7, 2023 at 19:52

dean glükler

1931 gold badge1 silver badge7 bronze badges

Collectives™ on Stack Overflow

How to find and replace a series of repeated words in an array of string?

4 Answers 4

1 Comment

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related