Split Text into Chunks

This function splits a text string into a vector of strings with a specified number of tokens each.

Usage

split_text(text, N)

Arguments

text

A character vector containing the text to be split.

N

An integer specifying the number of tokens per chunk.

Value

A character vector containing the chunks of text with N tokens each.

Examples

large_text <- "This is an example of a large text string
that will be split into chunks of N tokens each by our custom R function."
num_tokens_per_chunk <- 5
split_text(large_text, num_tokens_per_chunk)
#> [[1]]
#> [1] "This is an example of"
#> 
#> [[2]]
#> [1] "a large text string that"
#> 
#> [[3]]
#> [1] "will be split into chunks"
#> 
#> [[4]]
#> [1] "of N tokens each by"
#> 
#> [[5]]
#> [1] "our custom R function."
#>