Multimodal AI Intermediario ~30 min

Speech-to-Text

Topico 35 de 35

stt audio transcricao

AÇÕES

Progresso do Topico 0%

Speech-to-Text

No contexto de IA multimodal, tecnologia Speech-to-Text (STT) converte linguagem falada em texto escrito, permitindo integracao com outros tipos de dados como imagens e texto. Isso permite que sistemas de IA processem entrada de audio e combinem com informacoes visuais ou textuais.

O Que e Speech-to-Text?#

STT, tambem conhecido como ASR (Automatic Speech Recognition), e o processo de:

Captura: Receber audio de fala
Processamento: Analisar padroes acusticos
Transcricao: Converter em texto escrito
Pos-processamento: Adicionar pontuacao, formatacao

Como Funciona?#

Pipeline Tradicional#

Audio ──> Pre-processamento ──> Modelo Acustico ──> Modelo de Linguagem ──> Texto
          (ruido, normalizacao)  (fonemas)          (palavras/frases)

Modelos End-to-End Modernos#

Audio ──> Encoder (Transformer) ──> Decoder ──> Texto
          (representacao latente)   (geracao)

APIs e Servicos#

OpenAI Whisper API#

Python

from openai import OpenAI

client = OpenAI()

# Transcricao basica
with open("audio.mp3", "rb") as audio_file:
    transcript = client.audio.transcriptions.create(
        model="whisper-1",
        file=audio_file
    )

print(transcript.text)

Whisper com Opcoes Avancadas#

Python

# Transcricao com timestamps
transcript = client.audio.transcriptions.create(
    model="whisper-1",
    file=audio_file,
    response_format="verbose_json",
    timestamp_granularities=["word", "segment"]
)

# Acessar timestamps
for segment in transcript.segments:
    print(f"[{segment.start:.2f}s] {segment.text}")

Traducao de Audio#

Python

# Traduzir audio em qualquer idioma para ingles
translation = client.audio.translations.create(
    model="whisper-1",
    file=audio_file
)

Google Cloud Speech-to-Text#

Python

from google.cloud import speech

client = speech.SpeechClient()

with open("audio.wav", "rb") as audio_file:
    content = audio_file.read()

audio = speech.RecognitionAudio(content=content)
config = speech.RecognitionConfig(
    encoding=speech.RecognitionConfig.AudioEncoding.LINEAR16,
    sample_rate_hertz=16000,
    language_code="pt-BR",
    enable_automatic_punctuation=True
)

response = client.recognize(config=config, audio=audio)

for result in response.results:
    print(result.alternatives[0].transcript)

AWS Transcribe#

Python

import boto3

transcribe = boto3.client('transcribe')

# Iniciar job de transcricao
transcribe.start_transcription_job(
    TranscriptionJobName='meu-job',
    Media={'MediaFileUri': 's3://bucket/audio.mp3'},
    MediaFormat='mp3',
    LanguageCode='pt-BR'
)

Whisper Local (Open Source)#

Python

import whisper

# Carregar modelo
model = whisper.load_model("base")  # tiny, base, small, medium, large

# Transcrever
result = model.transcribe("audio.mp3")
print(result["text"])

# Com deteccao de idioma
result = model.transcribe("audio.mp3", task="transcribe")
print(f"Idioma detectado: {result['language']}")
print(f"Texto: {result['text']}")

Tamanhos de Modelo Whisper#

Modelo	Parametros	VRAM	Velocidade
tiny	39M	~1GB	Muito rapido
base	74M	~1GB	Rapido
small	244M	~2GB	Moderado
medium	769M	~5GB	Lento
large	1550M	~10GB	Muito lento

Integracao Multimodal#

Transcricao de Video#

Python

import moviepy.editor as mp
import whisper

# Extrair audio do video
video = mp.VideoFileClip("video.mp4")
video.audio.write_audiofile("audio.wav")

# Transcrever
model = whisper.load_model("base")
result = model.transcribe("audio.wav")

# Combinar com analise visual
# ... analise de frames do video

Assistente de Voz Multimodal#

Python

async def process_voice_with_context(audio_file, image_file=None):
    # 1. Transcrever audio
    transcript = await transcribe(audio_file)

    # 2. Se houver imagem, incluir na analise
    if image_file:
        response = await multimodal_analysis(
            text=transcript,
            image=image_file
        )
    else:
        response = await text_analysis(transcript)

    return response

Casos de Uso#

1. Legendagem Automatica#

Python

def generate_srt(audio_path):
    result = model.transcribe(audio_path, word_timestamps=True)

    srt_content = ""
    for i, segment in enumerate(result["segments"]):
        start = format_timestamp(segment["start"])
        end = format_timestamp(segment["end"])
        text = segment["text"]
        srt_content += f"{i+1}\n{start} --> {end}\n{text}\n\n"

    return srt_content

2. Resumo de Reunioes#

Python

def summarize_meeting(audio_path):
    # Transcrever
    transcript = transcribe(audio_path)

    # Resumir com LLM
    summary = llm.generate(f"""
    Resuma os pontos principais desta reuniao:

    {transcript}

    Inclua: decisoes tomadas, acoes definidas, proximos passos.
    """)

    return summary

3. Analise de Call Center#

Python

def analyze_call(audio_path):
    transcript = transcribe(audio_path)

    analysis = llm.generate(f"""
    Analise esta ligacao de atendimento:

    {transcript}

    Identifique:
    - Sentimento do cliente
    - Problema reportado
    - Solucao oferecida
    - Satisfacao final
    """)

    return analysis