Commit a5608e6f authored by Saliou's avatar Saliou

Initial commit

parents
File added
File added
.env
# Created by https://www.toptal.com/developers/gitignore/api/flask
# Edit at https://www.toptal.com/developers/gitignore?templates=flask
### Flask ###
instance/*
!instance/.gitignore
.webassets-cache
.env
### Flask.Python Stack ###
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# End of https://www.toptal.com/developers/gitignore/api/flask
\ No newline at end of file
before_script:
- docker version
- docker-compose version
build:
stage: build
tags:
- baamtu
script:
- docker build -t baamtu/demo_article_api:master .
- docker push baamtu/demo_article_api:master
pipeline {
agent any
environment {
registry = "baamtu/demo_article_api"
registryCredential = 'dockerhub'
dockerImage = ''
}
stages {
stage('Build') {
steps {
echo 'Building..'
}
}
stage('Test') {
steps {
echo 'Testing..'
}
}
stage('Build and push image') {
steps {
script {
echo 'Build..'
def version = readFile('VERSION')
def versions = version.split('\\.')
def major = versions[0]
def minor = versions[0] + '.' + versions[1]
def patch = version.trim()
docker.withRegistry('', 'dockerhub') {
def image = docker.build('baamtu/demo_article_api:dev',' -f ./Dockerfile .')
image.push()
}
}
}
}
stage('Cleaning up') {
steps {
sh "yes | docker system prune -a"
}
}
}
}
pipeline {
agent any
environment {
registry = "baamtu/demo_article_api"
registryCredential = 'dockerhub'
dockerImage = ''
}
stages {
stage('Build') {
steps {
echo 'Building..'
}
}
stage('Test') {
steps {
echo 'Testing..'
}
}
stage('Build and push image') {
steps {
script {
echo 'Build..'
def version = readFile('VERSION')
def versions = version.split('\\.')
def major = versions[0]
def minor = versions[0] + '.' + versions[1]
def patch = version.trim()
docker.withRegistry('', 'dockerhub') {
def image = docker.build('baamtu/demo_article_api:master',' -f ./Dockerfile .')
image.push()
}
}
}
}
stage('Cleaning up') {
steps {
sh "yes | docker system prune -a"
}
}
}
}
FROM amancevice/pandas:1.2.4-alpine
RUN apk add gcc musl-dev python3-dev libffi-dev openssl-dev
COPY . /app
WORKDIR /app
RUN pip install -r requirements.txt
EXPOSE 80
# ENTRYPOINT ["python"]
# CMD ["rest.py"]
CMD ["sh","-c","./gunicorn.sh"]
\ No newline at end of file
<img align="center" src=baamtu.jpeg style="width:150px">
<h1 align="center"> You Know You Share </h1>
<h3 align="center">Saliou FALL</h3>
---------------------------------------------------------------------------------------------
|
<h3 align="center"> DevOps Enginer / Baamtu </h3>
<h4 align="center">Email : saliou.fall@baamtu.com</h4>
------------------
## Task
In a perspective of integrating a recommendation system into a shop application module on . We have as a challenge to propose and implement an API based on python allowing in initially to make a random selection of the different product published by the users according to different parameters as input.
## Principle
The principle is the following: implementing an API that will return a list of random indexes of the different offers according to the parameters entered.
## Steps
<ol>
<li> Making a API call to retrieve data from the database </li>
<li> Pre-proccessing state ( cleaning ) </li>
<li> randomSelect function takes as input the dataset of article service , the selection parameter.</li>
<li> randomSelect function will makes a groupby according to the parameter</li>
<li> Random selection of one element on each group. </li>
<li> Remove the selected elements from the dataset</li>
<li> Restart from 4 as long as the dataset is not empty.</li>
<li> Return the index list. </li>
</ol>
from flask import Flask, g
from flask_httpauth import HTTPTokenAuth
app = Flask(__name__)
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
apiVersion: v2
name: api
description: A Helm chart for Kubernetes
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "1.16.0"
1. Get the application URL by running these commands:
{{- if .Values.ingress.enabled }}
{{- range $host := .Values.ingress.hosts }}
{{- range .paths }}
http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
{{- end }}
{{- end }}
{{- else if contains "NodePort" .Values.service.type }}
export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "api.fullname" . }})
export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
echo http://$NODE_IP:$NODE_PORT
{{- else if contains "LoadBalancer" .Values.service.type }}
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "api.fullname" . }}'
export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "api.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
echo http://$SERVICE_IP:{{ .Values.service.port }}
{{- else if contains "ClusterIP" .Values.service.type }}
export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "api.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
{{- end }}
{{/*
Expand the name of the chart.
*/}}
{{- define "api.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "api.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "api.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "api.labels" -}}
helm.sh/chart: {{ include "api.chart" . }}
{{ include "api.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "api.selectorLabels" -}}
app.kubernetes.io/name: {{ include "api.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
{{/*
Create the name of the service account to use
*/}}
{{- define "api.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "api.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "api.fullname" . }}
labels:
{{- include "api.labels" . | nindent 4 }}
spec:
{{- if not .Values.autoscaling.enabled }}
replicas: {{ .Values.replicaCount }}
{{- end }}
selector:
matchLabels:
{{- include "api.selectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "api.selectorLabels" . | nindent 8 }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "api.serviceAccountName" . }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- name: http
containerPort: 80
protocol: TCP
livenessProbe:
httpGet:
path: /
port: http
readinessProbe:
httpGet:
path: /
port: http
resources:
{{- toYaml .Values.resources | nindent 12 }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- if .Values.autoscaling.enabled }}
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
name: {{ include "api.fullname" . }}
labels:
{{- include "api.labels" . | nindent 4 }}
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: {{ include "api.fullname" . }}
minReplicas: {{ .Values.autoscaling.minReplicas }}
maxReplicas: {{ .Values.autoscaling.maxReplicas }}
metrics:
{{- if .Values.autoscaling.targetCPUUtilizationPercentage }}
- type: Resource
resource:
name: cpu
targetAverageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }}
{{- end }}
{{- if .Values.autoscaling.targetMemoryUtilizationPercentage }}
- type: Resource
resource:
name: memory
targetAverageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }}
{{- end }}
{{- end }}
{{- if .Values.ingress.enabled -}}
{{- $fullName := include "api.fullname" . -}}
{{- $svcPort := .Values.service.port -}}
{{- if and .Values.ingress.className (not (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion)) }}
{{- if not (hasKey .Values.ingress.annotations "kubernetes.io/ingress.class") }}
{{- $_ := set .Values.ingress.annotations "kubernetes.io/ingress.class" .Values.ingress.className}}
{{- end }}
{{- end }}
{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion -}}
apiVersion: networking.k8s.io/v1
{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
apiVersion: networking.k8s.io/v1beta1
{{- else -}}
apiVersion: extensions/v1beta1
{{- end }}
kind: Ingress
metadata:
name: {{ $fullName }}
labels:
{{- include "api.labels" . | nindent 4 }}
{{- with .Values.ingress.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
{{- if and .Values.ingress.className (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) }}
ingressClassName: {{ .Values.ingress.className }}
{{- end }}
{{- if .Values.ingress.tls }}
tls:
{{- range .Values.ingress.tls }}
- hosts:
{{- range .hosts }}
- {{ . | quote }}
{{- end }}
secretName: {{ .secretName }}
{{- end }}
{{- end }}
rules:
{{- range .Values.ingress.hosts }}
- host: {{ .host | quote }}
http:
paths:
{{- range .paths }}
- path: {{ .path }}
{{- if and .pathType (semverCompare ">=1.18-0" $.Capabilities.KubeVersion.GitVersion) }}
pathType: {{ .pathType }}
{{- end }}
backend:
{{- if semverCompare ">=1.19-0" $.Capabilities.KubeVersion.GitVersion }}
service:
name: {{ $fullName }}
port:
number: {{ $svcPort }}
{{- else }}
serviceName: {{ $fullName }}
servicePort: {{ $svcPort }}
{{- end }}
{{- end }}
{{- end }}
{{- end }}
apiVersion: v1
kind: Service
metadata:
name: {{ include "api.fullname" . }}
labels:
{{- include "api.labels" . | nindent 4 }}
spec:
type: {{ .Values.service.type }}
ports:
- port: {{ .Values.service.port }}
targetPort: http
protocol: TCP
name: http
selector:
{{- include "api.selectorLabels" . | nindent 4 }}
{{- if .Values.serviceAccount.create -}}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "api.serviceAccountName" . }}
labels:
{{- include "api.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}
apiVersion: v1
kind: Pod
metadata:
name: "{{ include "api.fullname" . }}-test-connection"
labels:
{{- include "api.labels" . | nindent 4 }}
annotations:
"helm.sh/hook": test
spec:
containers:
- name: wget
image: busybox
command: ['wget']
args: ['{{ include "api.fullname" . }}:{{ .Values.service.port }}']
restartPolicy: Never
# Default values for api.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
replicaCount: 1
image:
repository: nginx
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: ""
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
serviceAccount:
# Specifies whether a service account should be created
create: true
# Annotations to add to the service account
annotations: {}
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: ""
podAnnotations: {}
podSecurityContext: {}
# fsGroup: 2000
securityContext: {}
# capabilities:
# drop:
# - ALL
# readOnlyRootFilesystem: true
# runAsNonRoot: true
# runAsUser: 1000
service:
type: ClusterIP
port: 80
ingress:
enabled: false
className: ""
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
hosts:
- host: chart-example.local
paths:
- path: /
pathType: ImplementationSpecific
tls: []
# - secretName: chart-example-tls
# hosts:
# - chart-example.local
resources: {}
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
# limits:
# cpu: 100m
# memory: 128Mi
# requests:
# cpu: 100m
# memory: 128Mi
autoscaling:
enabled: false
minReplicas: 1
maxReplicas: 100
targetCPUUtilizationPercentage: 80
# targetMemoryUtilizationPercentage: 80
nodeSelector: {}
tolerations: []
affinity: {}
from app import app
from flaskext.mysql import MySQL
import os
from dotenv import load_dotenv
project_folder = os.getcwd()
load_dotenv(os.path.join(project_folder, '.env'))
mysql = MySQL()
# MySQL configurations
app.config['MYSQL_DATABASE_USER'] = os.getenv("MYSQL_DATABASE_USER")
app.config['MYSQL_DATABASE_PASSWORD'] = os.getenv("MYSQL_DATABASE_PASSWORD")
app.config['MYSQL_DATABASE_DB'] = os.getenv("MYSQL_DATABASE_DB")
app.config['MYSQL_DATABASE_HOST'] = os.getenv("MYSQL_DATABASE_HOST")
mysql.init_app(app)
\ No newline at end of file
version: '3.5'
networks:
mysql:
external: false
internal:
external: false
services:
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
networks:
- internal
volumes:
- mysql:/var/lib/mysql
environment:
MYSQL_DATABASE: demo
MYSQL_ROOT_PASSWORD: admin
MYSQL_USER: admin
MYSQL_PASSWORD: admin
phpmyadmin:
image: phpmyadmin
restart: always
networks:
- internal
- mysql
ports:
- "8080:80"
environment:
PMA_HOST: db
PMA_PORT: 3306
api:
build: .
restart: always
networks:
- internal
- mysql
volumes:
- .:/app
ports:
- "8000:80"
volumes:
mysql:
\ No newline at end of file
__pycache__/
.env
\ No newline at end of file
TOKEN=bskjks34442i
USER_TOKEN=user0003
API_URL=http://localhost:5000
MYSQL_DATABASE_USER=root
MYSQL_DATABASE_PASSWORD=admin
MYSQL_DATABASE_DB=database
MYSQL_DATABASE_HOST=db
#!/bin/sh
gunicorn rest:app -w 2 --threads 2 -b 0.0.0.0:80 # --chdir app
\ No newline at end of file
flask-mysql
pymysql
werkzeug
flask_httpauth
numpy
requests
fsspec
python-dotenv
gunicorn
\ No newline at end of file
import pymysql
from app import app
from db import mysql
from flask import jsonify,request,json
from flask_httpauth import HTTPBasicAuth
from flask_httpauth import HTTPTokenAuth
import decimal
import pandas as pd
import numpy as np
import functools
import operator
import requests
import json
from requests.structures import CaseInsensitiveDict
import os
from dotenv import load_dotenv
project_folder = os.getcwd()
load_dotenv(os.path.join(project_folder, '.env'))
class DecimalEncoder(json.JSONEncoder):
def default(self, o):
if isinstance(o, decimal.Decimal):
return '`'+str(o)+'`' #` is special, will be removed later
return super(DecimalEncoder, self).default(o)
auth = HTTPTokenAuth(scheme='Bearer')
tokens = {
os.getenv("TOKEN"): os.getenv("USER_TOKEN"),
}
headers = {'Authorization': 'Bearer '+ os.getenv("TOKEN") }
def randomSelect(data,group ,size=1, replace=False):
index=[]
fn = lambda obj: obj.loc[np.random.choice(obj.index, 1 , replace),:]
while data.empty != True:
randomListByUser=data.groupby(group, as_index=False).apply(fn)
index.append( [ item for item in randomListByUser.index.get_level_values("id") ] )
data=data.drop(randomListByUser.index.get_level_values("id"), axis=0)
return functools.reduce(operator.iconcat, index, [])
@auth.verify_token
def verify_token(token):
if token in tokens:
return tokens[token]
@app.route('/')
@auth.login_required
def index():
return "Hello, {}!".format(auth.current_user())
@app.route('/articles')
@auth.login_required
def articles():
conn = mysql.connect()
cursor = conn.cursor(pymysql.cursors.DictCursor)
cursor.execute("SELECT id,entreprise_id,category_id,user_id,created_at,localite,vues,secteur FROM articles")
rows = cursor.fetchall()
resp = jsonify(rows)
resp.status_code = 200
return resp
@app.route('/get_index')
@auth.login_required
def get_index():
data = pd.read_json( json.dumps(requests.get(os.getenv("API_URL")+'/articles', headers=headers).json())).set_index("id")
size = 1 # sample size
replace = False # with replacement
return json.dumps( randomSelect(data, request.args.get('group'), size, replace))
if __name__ == "__main__":
app.run(debug=True,host='0.0.0.0')
{
"cells": [
{
"cell_type": "markdown",
"id": "7d5d99b9",
"metadata": {},
"source": [
"\n",
"<img align=\"center\" src=EVA.png style=\"width:300px\"> \n",
"<h1 align=\"center\"> API for services offers </h1>\n",
"<h3 align=\"center\">Saliou FALL</h3>\n",
"\n",
"---------------------------------------------------------------------------------------------\n",
"\n",
"<h3 align=\"center\"> Security Architect / EVA SENEGAL</h3>\n",
"<h4 align=\"center\">Email : saliou.fall@eva.sn</h4>\n",
"\n",
"------------------\n",
"\n",
"\n",
"\n",
"## Task \n",
"\n",
"In a perspective of integrating a recommendation system in the service offering module on Evablis Plas. We have as a challenge to propose and implement an API based on python allowing in initially to make a random selection of the different offers published by the users according to different parameters as input.\n",
"\n",
"## Principle\n",
"The principle is the following: implementing an API that will return a list of random indexes of the different offers according to the parameters entered.\n",
"\n",
"## Steps\n",
"\n",
"<ol>\n",
"<li> Making a API call to retrieve data from the database </li>\n",
"<li> Pre-proccessing state ( cleaning ) </li>\n",
"<li> randomSelect function takes as input the dataset of the service offers, the selection parameter.</li>\n",
"<li> randomSelect function will makes a groupby according to the parameter</li>\n",
"<li> Random selection of one element on each group. </li>\n",
"<li> Remove the selected elements from the dataset</li>\n",
"<li> Restart from 4 as long as the dataset is not empty.</li>\n",
"<li> Return the index list. </li>\n",
"</ol>\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 44,
"id": "9b14faed",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>category_id</th>\n",
" <th>created_at</th>\n",
" <th>entreprise_id</th>\n",
" <th>localite</th>\n",
" <th>user_id</th>\n",
" </tr>\n",
" <tr>\n",
" <th>id</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>NaN</td>\n",
" <td>2020-12-14 13:48:28+00:00</td>\n",
" <td>14</td>\n",
" <td>Dakar</td>\n",
" <td>14</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>NaN</td>\n",
" <td>2020-12-14 20:17:07+00:00</td>\n",
" <td>21</td>\n",
" <td>Dakar</td>\n",
" <td>21</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>NaN</td>\n",
" <td>2020-12-14 20:25:38+00:00</td>\n",
" <td>21</td>\n",
" <td>Dakar</td>\n",
" <td>21</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>NaN</td>\n",
" <td>2020-12-14 20:37:54+00:00</td>\n",
" <td>21</td>\n",
" <td>Dakar</td>\n",
" <td>21</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6</th>\n",
" <td>NaN</td>\n",
" <td>2020-12-14 20:49:30+00:00</td>\n",
" <td>21</td>\n",
" <td>Dakar</td>\n",
" <td>21</td>\n",
" </tr>\n",
" <tr>\n",
" <th>...</th>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" <td>...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>74</th>\n",
" <td>NaN</td>\n",
" <td>2021-06-19 09:18:22+00:00</td>\n",
" <td>45</td>\n",
" <td>Dakar</td>\n",
" <td>45</td>\n",
" </tr>\n",
" <tr>\n",
" <th>75</th>\n",
" <td>NaN</td>\n",
" <td>2021-06-19 09:20:51+00:00</td>\n",
" <td>45</td>\n",
" <td>Dakar</td>\n",
" <td>45</td>\n",
" </tr>\n",
" <tr>\n",
" <th>76</th>\n",
" <td>NaN</td>\n",
" <td>2021-06-19 09:54:30+00:00</td>\n",
" <td>45</td>\n",
" <td>Dakar</td>\n",
" <td>45</td>\n",
" </tr>\n",
" <tr>\n",
" <th>77</th>\n",
" <td>NaN</td>\n",
" <td>2021-07-07 00:07:22+00:00</td>\n",
" <td>122</td>\n",
" <td>Dakar</td>\n",
" <td>122</td>\n",
" </tr>\n",
" <tr>\n",
" <th>78</th>\n",
" <td>NaN</td>\n",
" <td>2021-07-07 00:07:24+00:00</td>\n",
" <td>122</td>\n",
" <td>Dakar</td>\n",
" <td>122</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>75 rows × 5 columns</p>\n",
"</div>"
],
"text/plain": [
" category_id created_at entreprise_id localite user_id\n",
"id \n",
"2 NaN 2020-12-14 13:48:28+00:00 14 Dakar 14\n",
"3 NaN 2020-12-14 20:17:07+00:00 21 Dakar 21\n",
"4 NaN 2020-12-14 20:25:38+00:00 21 Dakar 21\n",
"5 NaN 2020-12-14 20:37:54+00:00 21 Dakar 21\n",
"6 NaN 2020-12-14 20:49:30+00:00 21 Dakar 21\n",
".. ... ... ... ... ...\n",
"74 NaN 2021-06-19 09:18:22+00:00 45 Dakar 45\n",
"75 NaN 2021-06-19 09:20:51+00:00 45 Dakar 45\n",
"76 NaN 2021-06-19 09:54:30+00:00 45 Dakar 45\n",
"77 NaN 2021-07-07 00:07:22+00:00 122 Dakar 122\n",
"78 NaN 2021-07-07 00:07:24+00:00 122 Dakar 122\n",
"\n",
"[75 rows x 5 columns]"
]
},
"execution_count": 44,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#import the pandas library\n",
"import pandas as pd\n",
"import numpy as np\n",
"import functools\n",
"import operator\n",
"import requests\n",
"import json \n",
"\n",
"from requests.structures import CaseInsensitiveDict\n",
"\n",
"# headers[\"Accept\"] = \"application/json\"\n",
"url = \"https://api.eva.sn/offres\"\n",
"# url = \"http://localhost:8000/offres\"\n",
"headers = {'Authorization': 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXV'}\n",
"data = pd.read_json( json.dumps(requests.get(url, headers=headers).json())).set_index(\"id\")\n",
"df = pd.DataFrame(data)\n",
"# data=df.drop(columns=['slug', 'descriptif', 'attachment','updated_at','status','vues'])\n",
"# data=data.set_index(\"id\")\n",
"data"
]
},
{
"cell_type": "code",
"execution_count": 34,
"id": "2d799667",
"metadata": {},
"outputs": [],
"source": [
"def randomSelect(data,group ,size, replace):\n",
" index=[]\n",
" fn = lambda obj: obj.loc[np.random.choice(obj.index, 1 , replace),:]\n",
" while data.empty != True:\n",
" randomListByUser=data.groupby(group, as_index=False).apply(fn)\n",
" index.append( [ item for item in randomListByUser.index.get_level_values(\"id\") ] )\n",
" data=data.drop(randomListByUser.index.get_level_values(\"id\"), axis=0)\n",
" return functools.reduce(operator.iconcat, index, []) "
]
},
{
"cell_type": "code",
"execution_count": 42,
"id": "a851a487",
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/plain": [
"[1, 24, 11, 6, 16, 2, 25, 10, 12, 17, 22, 9, 14, 15, 3, 8, 13, 4, 5, 7]"
]
},
"execution_count": 42,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"\n",
"size = 1 # sample size\n",
"replace = False # with replacement\n",
"group='user_id'\n",
"json.dumps(randomSelect(data, group, size, replace))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cf9d7039",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.8"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
id,title,slug,descriptif,localite,category_id,entreprise_id,secteur,status,user_id,attachment,created_at,updated_at,vues
1,Maecenas sit amet mi nec magna porta consectetur eu ac odio,maecenas-sit-amet-mi-nec-magna-porta-consectetur-eu-ac-odio,"<p><span style=""color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif; text-align: justify;"">Duis pretium diam a felis pretium, a rutrum nibh pretium. Vestibulum porta rutrum turpis. Maecenas sit amet mi nec magna porta consectetur eu ac odio. Phasellus quis justo gravida justo tincidunt interdum. Aenean tristique erat varius ante efficitur imperdiet. Vivamus magna massa, luctus id congue in, sollicitudin at sem. Pellentesque cursus vestibulum purus, vitae elementum nisi aliquam in. Phasellus scelerisque, justo eu molestie ultrices, ipsum lacus condimentum orci, id porttitor orci nisl at libero. Nullam semper lorem nisi, aliquam consequat odio fringilla et.&nbsp;</span></p><p><span style=""color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif; text-align: justify;"">Etiam id turpis vel dolor ornare lacinia. Quisque pharetra, enim ut laoreet luctus, risus metus congue nibh, faucibus fermentum lectus ex a nisi. Donec tristique libero consequat lorem efficitur, vitae tristique felis varius. Fusce tristique augue nec risus convallis, vitae efficitur est varius. Morbi tempor leo non rutrum euismod. Donec iaculis nisi quis purus feugiat auctor.</span><br></p>",Dakar,7,1,Informatique et télécoms,1,1,"{""path"": ""uploads/offres/qMQhmmdrOeDWbcZE7enmZJHMx.jpg"", ""type"": ""image""}",2020-08-26 15:54:36,2020-09-08 16:09:59,"0"
2,"Maecenas eget pharetra nulla, eget tempus nibh",maecenas-eget-pharetra-nulla-eget-tempus-nibh,"<p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px; text-align: justify; color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif;"">Quisque nulla quam, pulvinar sit amet luctus eu, rhoncus ac ex. Sed dapibus bibendum magna vulputate mollis. Curabitur non urna blandit, congue tellus eu, feugiat ante.&nbsp;</p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px; text-align: justify; color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif;""><ul><li>Nulla quis tellus vitae massa fringilla sagittis.&nbsp;</li><li>Vivamus sapien velit, consequat</li><li>&nbsp;maximus at, finibus pharetra purus.</li></ul></p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px; text-align: justify; color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif;"">&nbsp;Pellentesque id turpis in mi aliquet pulvinar. Quisque rhoncus orci et elementum pulvinar. Donec scelerisque eros eget tristique tempus. Proin facilisis orci in enim porttitor, nec pulvinar urna vulputate. Morbi fringilla massa id sapien gravida, eu vestibulum sem bibendum. Nulla eleifend viverra enim non sodales. Phasellus maximus placerat dolor nec luctus. Sed tincidunt mauris ligula, non accumsan nunc laoreet sed. Maecenas eget pharetra nulla, eget tempus nibh.</p><p><br></p>",Dakar,9,1,Communication - Marketing - Pub,1,1,"{""path"": ""uploads/offres/LmJ6933bCP7tQM4405leZuh9r.jpg"", ""type"": ""image""}",2020-08-26 16:20:03,2020-08-26 16:20:03,"0"
3,Eu gravida orci suscipit. Donec ac nibh ac justo rutrum tempus,eu-gravida-orci-suscipit-donec-ac-nibh-ac-justo-rutrum-tempus,"<p id=""lipsum"" style=""margin: 0px; padding: 0px; text-align: justify;""></p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">In hac habitasse platea dictumst. Aliquam ac dui odio. Etiam euismod sit amet tortor eget tincidunt. Sed eu ante dapibus ipsum pellentesque congue id sit amet arcu. Phasellus eget risus consectetur, imperdiet lorem eu, sollicitudin diam. Nam eu enim leo. Fusce ultrices dictum ligula auctor auctor. Morbi vestibulum tristique quam nec imperdiet.</p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">Donec sagittis dui at facilisis mattis. Maecenas et lacus eros. Nulla vehicula elit diam, eget euismod nisl aliquam nec. Duis suscipit sapien nec enim consectetur, quis rutrum sapien consectetur. Nulla in gravida nisl, at fringilla urna. Maecenas at ornare nisl. Nam lacinia faucibus urna, in ultrices lacus sagittis et. Pellentesque hendrerit vehicula lorem, at ultricies lacus. Aliquam dictum magna vitae erat ornare, quis commodo ante dapibus. Nunc eget blandit ipsum. Quisque congue massa a diam imperdiet fermentum. Aliquam pulvinar vehicula arcu, non suscipit metus pulvinar vitae.</p><p style=""color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif;""><br></p><p></p>",Dakar,8,2,Informatique et télécoms,1,2,"{""path"": ""uploads/offres/UyBMrfL8Wj2GnQ82wJf70aJww.jpg"", ""type"": ""image""}",2020-08-31 12:44:32,2020-08-31 12:44:32,"0"
4,Quisque ornare sollicitudin libero eget pulvinar pellentesque convallis neque sit amet urna vestibulum commodo,quisque-ornare-sollicitudin-libero-eget-pulvinar-pellentesque-convallis-neque-sit-amet-urna-vestibulum-commodo,"<p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">Sed malesuada, ante sit amet ultrices iaculis, est justo scelerisque nisi, quis vehicula nibh lorem et dolor. Nunc ut elementum mauris. Morbi in erat at nibh convallis efficitur. Ut blandit urna quis eleifend auctor. Proin laoreet sagittis urna non consequat. Quisque viverra cursus ullamcorper. Ut accumsan sem at aliquet bibendum. Cras molestie tempus metus, ac efficitur lorem porta vitae.</p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">Donec vitae sagittis felis, et fermentum dolor. Nullam eleifend metus nec ante malesuada finibus. Nulla varius egestas ante, ut ullamcorper ante suscipit ac. Quisque ornare posuere dictum. Morbi dictum velit sed sapien fringilla, at viverra tellus malesuada. In tincidunt felis ut mauris feugiat, a auctor risus accumsan. Mauris faucibus odio lacus, quis lacinia nulla tincidunt at. Praesent ligula justo, pretium sed nisi in, hendrerit congue enim. Fusce volutpat quam libero, at rhoncus dui laoreet ut. Phasellus porta pretium augue, et consectetur nisl consequat condimentum. Suspendisse potenti. Sed ornare vulputate ante, semper ornare tellus iaculis et. Proin lacinia laoreet eros sit amet congue. Aenean euismod tristique lobortis. Praesent quis ex at nisi egestas elementum. Suspendisse efficitur dapibus est, eu lobortis mi mollis ut.</p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">In interdum eros ante, eget mattis mi vulputate sed. Nulla interdum viverra accumsan. Quisque vel leo posuere neque sagittis aliquet. Aenean porta magna eu erat dictum, feugiat cursus ex iaculis. Sed nec hendrerit nulla. Suspendisse potenti. Ut eu condimentum nulla. Duis non eros eu arcu tincidunt ullamcorper non a ligula. Suspendisse dictum faucibus ex, ac volutpat libero auctor in. Etiam scelerisque dolor vehicula velit pellentesque elementum.</p><p><br></p>",Dakar,9,2,Informatique et télécoms,1,2,"{""path"": ""uploads/offres/rTuVxWTg7ArfBjyMonns3o6cr.png"", ""type"": ""image""}",2020-09-08 10:39:35,2020-09-08 10:39:35,"0"
5,Cotizel,cotizel,"Cotizel est une solution digital d'encaissement via mobile money.
A travers son CRM et son système d'integration, elle permet aux opérateurs économiques d'encaisser des flux de paiements, uniques ou récurrent via mobile money ou carte bancaire, sans contraintes technologiques.",Dakar,7,5,Communication - Marketing - Pub,1,5,"{""path"":""uploads\/offres\/4AKSMLept5zxZjMufAYmBV1VQ.png"",""type"":""image""}",2020-10-23 11:10:55,2020-12-01 18:53:17,"0"
6,Services d'assurance,services-dassurance,Venez découvrir nos pack d'assurance à partir de 10 000FCFA,Kaolack,9,10,"Banque, assurance",1,10,"{""path"":""uploads\/offres\/yYqVYkAj215sXu7ElN0Bi4SLC.jpg"",""type"":""image""}",2020-10-28 12:06:43,2020-12-02 12:31:59,"0"
7,ouyouma,ouyouma,Accompagnement psychosocial pour les personnes physiques en détresse émotionnelle.,Dakar,7,5,Social,1,5,"{""path"":""uploads\/offres\/Itd5W7DomJ8BMVS8ITbPq9ZK4.png"",""type"":""image""}",2020-10-28 12:08:43,2020-10-28 12:08:51,"0"
8,hdhfd,hdhfd,hgfjdshndyjhnhkj,Dakar,7,5,"Audit, gestion",1,5,"{""path"":""uploads\/offres\/DV8YGpQWxbaIapC6mJipMSOew.jpg"",""type"":""image""}",2020-10-28 12:10:23,2020-10-28 12:12:28,"0"
9,ouyouma,ouyouma,accompagnement psychologique pour les pme,Dakar,7,5,Social,10,5,,2020-11-10 12:22:40,2020-11-10 12:30:40,"0"
10,accompagnement des pme de EVA,accompagnement-des-pme-de-eva,nous accompagnons les pme a se digitaliser de façon soft prenant en compte leur contrainte de temps et financière.,Dakar,"0",5,Informatique et télécoms,1,5,,2020-11-11 12:37:38,2020-11-11 12:37:38,"0"
11,accompagnement des pme,accompagnement-des-pme,Nous accompagnos les PME,Dakar,"0",5,"Audit, gestion",1,5,"{""path"":""uploads\/offres\/Moml4MwKRtnrIXJ3mr3s8jzcd.png"",""type"":""image""}",2020-11-30 17:25:15,2020-11-30 17:25:15,"0"
12,Chaussures en cuir,chaussures-en-cuir,Venez découvrir nos chaussures en cuir de Ngay Mékhé,Dakar,"0",10,"Mode, industrie textile",1,10,"{""path"":""uploads\/offres\/EW0kVdBHfyIqjv8iQTsJ2lQAj.jpg"",""type"":""image""}",2020-12-02 11:29:13,2020-12-02 12:29:20,"0"
13,Pagnes WAX,pagnes-wax,Superbes Pagnes WAX en promotion,Thiès,"0",10,"Mode, industrie textile",1,10,"{""path"":""uploads\/offres\/OADnCJ29QYrrPhDI5usBoV0Gj.jpg"",""type"":""image""}",2020-12-02 11:35:21,2020-12-02 12:28:49,"0"
14,Travaux BTP,travaux-btp,Bénéficiez des meilleures services BTP avec un tarif adapté,Fatick,"0",10,"BTP, architecture",1,10,"{""path"":""uploads\/offres\/CtqSfbLMMpeBI9U2mei36FdRn.jpg"",""type"":""image""}",2020-12-02 12:26:07,2020-12-02 12:26:07,"0"
15,Produits capillaires,produits-capillaires,Vente de produits capillaires sans sulfate,Dakar,"0",11,"Chimie, pharmacie",1,11,"{""path"":""uploads\/offres\/3UGRUD5Cg7THxSV59ebrcK7ut.jpg"",""type"":""image""}",2020-12-02 16:13:41,2020-12-02 16:13:41,"0"
16,Jus de fruits bio,jus-de-fruits-bio,Délicieux jus de fruits frais sans sucre ajouté,Dakar,"0",11,Industrie alimentaire,1,11,"{""path"":""uploads\/offres\/ciHsYHVAUrKsNni6jysoRe2pX.jpg"",""type"":""image""}",2020-12-02 16:16:34,2020-12-02 16:16:34,"0"
17,Jolis sacs personnalisés 2 pour le prix d'1,jolis-sacs-personnalises-2-pour-le-prix-d1,Sacs personnalisés en promo. Stock limité,Dakar,"0",11,"Mode, industrie textile",1,11,"{""path"":""uploads\/offres\/EUo7oK5NFih4Or1BRrxLquKwk.jpg"",""type"":""image""}",2020-12-02 16:20:17,2020-12-02 16:20:17,"0"
22,Praesent lobortis velit aliquam arcu pulvinar luctus 1,praesent-lobortis-velit-aliquam-arcu-pulvinar-luctus-1,Praesent lobortis velit aliquam arcu pulvinar luctus 1,Dakar,"0",2,Automobile,1,2,"{""path"":""uploads\/offres\/OVDYJwTMqD0GJshHBVCgTOveW.jpg"",""type"":""image""}",2021-02-12 14:27:02,2021-02-12 14:27:02,"0"
24,Praesent lobortis velit aliquam arcu pulvinar luctus 2,praesent-lobortis-velit-aliquam-arcu-pulvinar-luctus-2,Praesent lobortis velit aliquam arcu pulvinar luctus 2,Dakar,"0",2,Audiovisuel - Spectacle,1,2,"{""path"":""uploads\/offres\/Ker1ckPLrMKvyVVrjP0cLSkyn.png"",""type"":""image""}",2021-02-12 14:28:53,2021-02-12 14:28:53,"0"
25,Praesent lobortis velit aliquam arcu pulvinar luctus 3,praesent-lobortis-velit-aliquam-arcu-pulvinar-luctus-3,Praesent lobortis velit aliquam arcu pulvinar luctus 3,Dakar,"0",2,"Edition, Journalisme",1,2,"{""path"":""uploads\/offres\/gI4kfZm1d5WybMcrbKze2gWqQ.png"",""type"":""image""}",2021-02-12 14:29:36,2021-02-12 14:29:36,"0"
id,title,slug,descriptif,localite,category_id,entreprise_id,secteur,status,user_id,attachment,created_at,updated_at,vues
1,Maecenas sit amet mi nec magna porta consectetur eu ac odio,maecenas-sit-amet-mi-nec-magna-porta-consectetur-eu-ac-odio,"<p><span style=""color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif; text-align: justify;"">Duis pretium diam a felis pretium, a rutrum nibh pretium. Vestibulum porta rutrum turpis. Maecenas sit amet mi nec magna porta consectetur eu ac odio. Phasellus quis justo gravida justo tincidunt interdum. Aenean tristique erat varius ante efficitur imperdiet. Vivamus magna massa, luctus id congue in, sollicitudin at sem. Pellentesque cursus vestibulum purus, vitae elementum nisi aliquam in. Phasellus scelerisque, justo eu molestie ultrices, ipsum lacus condimentum orci, id porttitor orci nisl at libero. Nullam semper lorem nisi, aliquam consequat odio fringilla et.&nbsp;</span></p><p><span style=""color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif; text-align: justify;"">Etiam id turpis vel dolor ornare lacinia. Quisque pharetra, enim ut laoreet luctus, risus metus congue nibh, faucibus fermentum lectus ex a nisi. Donec tristique libero consequat lorem efficitur, vitae tristique felis varius. Fusce tristique augue nec risus convallis, vitae efficitur est varius. Morbi tempor leo non rutrum euismod. Donec iaculis nisi quis purus feugiat auctor.</span><br></p>",Dakar,7,1,Informatique et télécoms,1,1,"{""path"": ""uploads/offres/qMQhmmdrOeDWbcZE7enmZJHMx.jpg"", ""type"": ""image""}",2020-08-26 15:54:36,2020-09-08 16:09:59,"0"
2,"Maecenas eget pharetra nulla, eget tempus nibh",maecenas-eget-pharetra-nulla-eget-tempus-nibh,"<p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px; text-align: justify; color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif;"">Quisque nulla quam, pulvinar sit amet luctus eu, rhoncus ac ex. Sed dapibus bibendum magna vulputate mollis. Curabitur non urna blandit, congue tellus eu, feugiat ante.&nbsp;</p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px; text-align: justify; color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif;""><ul><li>Nulla quis tellus vitae massa fringilla sagittis.&nbsp;</li><li>Vivamus sapien velit, consequat</li><li>&nbsp;maximus at, finibus pharetra purus.</li></ul></p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px; text-align: justify; color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif;"">&nbsp;Pellentesque id turpis in mi aliquet pulvinar. Quisque rhoncus orci et elementum pulvinar. Donec scelerisque eros eget tristique tempus. Proin facilisis orci in enim porttitor, nec pulvinar urna vulputate. Morbi fringilla massa id sapien gravida, eu vestibulum sem bibendum. Nulla eleifend viverra enim non sodales. Phasellus maximus placerat dolor nec luctus. Sed tincidunt mauris ligula, non accumsan nunc laoreet sed. Maecenas eget pharetra nulla, eget tempus nibh.</p><p><br></p>",Dakar,9,1,Communication - Marketing - Pub,1,1,"{""path"": ""uploads/offres/LmJ6933bCP7tQM4405leZuh9r.jpg"", ""type"": ""image""}",2020-08-26 16:20:03,2020-08-26 16:20:03,"0"
3,Eu gravida orci suscipit. Donec ac nibh ac justo rutrum tempus,eu-gravida-orci-suscipit-donec-ac-nibh-ac-justo-rutrum-tempus,"<p id=""lipsum"" style=""margin: 0px; padding: 0px; text-align: justify;""></p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">In hac habitasse platea dictumst. Aliquam ac dui odio. Etiam euismod sit amet tortor eget tincidunt. Sed eu ante dapibus ipsum pellentesque congue id sit amet arcu. Phasellus eget risus consectetur, imperdiet lorem eu, sollicitudin diam. Nam eu enim leo. Fusce ultrices dictum ligula auctor auctor. Morbi vestibulum tristique quam nec imperdiet.</p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">Donec sagittis dui at facilisis mattis. Maecenas et lacus eros. Nulla vehicula elit diam, eget euismod nisl aliquam nec. Duis suscipit sapien nec enim consectetur, quis rutrum sapien consectetur. Nulla in gravida nisl, at fringilla urna. Maecenas at ornare nisl. Nam lacinia faucibus urna, in ultrices lacus sagittis et. Pellentesque hendrerit vehicula lorem, at ultricies lacus. Aliquam dictum magna vitae erat ornare, quis commodo ante dapibus. Nunc eget blandit ipsum. Quisque congue massa a diam imperdiet fermentum. Aliquam pulvinar vehicula arcu, non suscipit metus pulvinar vitae.</p><p style=""color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif;""><br></p><p></p>",Dakar,8,2,Informatique et télécoms,1,2,"{""path"": ""uploads/offres/UyBMrfL8Wj2GnQ82wJf70aJww.jpg"", ""type"": ""image""}",2020-08-31 12:44:32,2020-08-31 12:44:32,"0"
4,Quisque ornare sollicitudin libero eget pulvinar pellentesque convallis neque sit amet urna vestibulum commodo,quisque-ornare-sollicitudin-libero-eget-pulvinar-pellentesque-convallis-neque-sit-amet-urna-vestibulum-commodo,"<p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">Sed malesuada, ante sit amet ultrices iaculis, est justo scelerisque nisi, quis vehicula nibh lorem et dolor. Nunc ut elementum mauris. Morbi in erat at nibh convallis efficitur. Ut blandit urna quis eleifend auctor. Proin laoreet sagittis urna non consequat. Quisque viverra cursus ullamcorper. Ut accumsan sem at aliquet bibendum. Cras molestie tempus metus, ac efficitur lorem porta vitae.</p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">Donec vitae sagittis felis, et fermentum dolor. Nullam eleifend metus nec ante malesuada finibus. Nulla varius egestas ante, ut ullamcorper ante suscipit ac. Quisque ornare posuere dictum. Morbi dictum velit sed sapien fringilla, at viverra tellus malesuada. In tincidunt felis ut mauris feugiat, a auctor risus accumsan. Mauris faucibus odio lacus, quis lacinia nulla tincidunt at. Praesent ligula justo, pretium sed nisi in, hendrerit congue enim. Fusce volutpat quam libero, at rhoncus dui laoreet ut. Phasellus porta pretium augue, et consectetur nisl consequat condimentum. Suspendisse potenti. Sed ornare vulputate ante, semper ornare tellus iaculis et. Proin lacinia laoreet eros sit amet congue. Aenean euismod tristique lobortis. Praesent quis ex at nisi egestas elementum. Suspendisse efficitur dapibus est, eu lobortis mi mollis ut.</p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">In interdum eros ante, eget mattis mi vulputate sed. Nulla interdum viverra accumsan. Quisque vel leo posuere neque sagittis aliquet. Aenean porta magna eu erat dictum, feugiat cursus ex iaculis. Sed nec hendrerit nulla. Suspendisse potenti. Ut eu condimentum nulla. Duis non eros eu arcu tincidunt ullamcorper non a ligula. Suspendisse dictum faucibus ex, ac volutpat libero auctor in. Etiam scelerisque dolor vehicula velit pellentesque elementum.</p><p><br></p>",Dakar,9,2,Informatique et télécoms,1,2,"{""path"": ""uploads/offres/rTuVxWTg7ArfBjyMonns3o6cr.png"", ""type"": ""image""}",2020-09-08 10:39:35,2020-09-08 10:39:35,"0"
5,Cotizel,cotizel,"Cotizel est une solution digital d'encaissement via mobile money.
A travers son CRM et son système d'integration, elle permet aux opérateurs économiques d'encaisser des flux de paiements, uniques ou récurrent via mobile money ou carte bancaire, sans contraintes technologiques.",Dakar,7,5,Communication - Marketing - Pub,1,5,"{""path"":""uploads\/offres\/4AKSMLept5zxZjMufAYmBV1VQ.png"",""type"":""image""}",2020-10-23 11:10:55,2020-12-01 18:53:17,"0"
6,Services d'assurance,services-dassurance,Venez découvrir nos pack d'assurance à partir de 10 000FCFA,Kaolack,9,10,"Banque, assurance",1,10,"{""path"":""uploads\/offres\/yYqVYkAj215sXu7ElN0Bi4SLC.jpg"",""type"":""image""}",2020-10-28 12:06:43,2020-12-02 12:31:59,"0"
7,ouyouma,ouyouma,Accompagnement psychosocial pour les personnes physiques en détresse émotionnelle.,Dakar,7,5,Social,1,5,"{""path"":""uploads\/offres\/Itd5W7DomJ8BMVS8ITbPq9ZK4.png"",""type"":""image""}",2020-10-28 12:08:43,2020-10-28 12:08:51,"0"
8,hdhfd,hdhfd,hgfjdshndyjhnhkj,Dakar,7,5,"Audit, gestion",1,5,"{""path"":""uploads\/offres\/DV8YGpQWxbaIapC6mJipMSOew.jpg"",""type"":""image""}",2020-10-28 12:10:23,2020-10-28 12:12:28,"0"
9,ouyouma,ouyouma,accompagnement psychologique pour les pme,Dakar,7,5,Social,10,5,,2020-11-10 12:22:40,2020-11-10 12:30:40,"0"
10,accompagnement des pme de EVA,accompagnement-des-pme-de-eva,nous accompagnons les pme a se digitaliser de façon soft prenant en compte leur contrainte de temps et financière.,Dakar,"0",5,Informatique et télécoms,1,5,,2020-11-11 12:37:38,2020-11-11 12:37:38,"0"
11,accompagnement des pme,accompagnement-des-pme,Nous accompagnos les PME,Dakar,"0",5,"Audit, gestion",1,5,"{""path"":""uploads\/offres\/Moml4MwKRtnrIXJ3mr3s8jzcd.png"",""type"":""image""}",2020-11-30 17:25:15,2020-11-30 17:25:15,"0"
12,Chaussures en cuir,chaussures-en-cuir,Venez découvrir nos chaussures en cuir de Ngay Mékhé,Dakar,"0",10,"Mode, industrie textile",1,10,"{""path"":""uploads\/offres\/EW0kVdBHfyIqjv8iQTsJ2lQAj.jpg"",""type"":""image""}",2020-12-02 11:29:13,2020-12-02 12:29:20,"0"
13,Pagnes WAX,pagnes-wax,Superbes Pagnes WAX en promotion,Thiès,"0",10,"Mode, industrie textile",1,10,"{""path"":""uploads\/offres\/OADnCJ29QYrrPhDI5usBoV0Gj.jpg"",""type"":""image""}",2020-12-02 11:35:21,2020-12-02 12:28:49,"0"
14,Travaux BTP,travaux-btp,Bénéficiez des meilleures services BTP avec un tarif adapté,Fatick,"0",10,"BTP, architecture",1,10,"{""path"":""uploads\/offres\/CtqSfbLMMpeBI9U2mei36FdRn.jpg"",""type"":""image""}",2020-12-02 12:26:07,2020-12-02 12:26:07,"0"
15,Produits capillaires,produits-capillaires,Vente de produits capillaires sans sulfate,Dakar,"0",11,"Chimie, pharmacie",1,11,"{""path"":""uploads\/offres\/3UGRUD5Cg7THxSV59ebrcK7ut.jpg"",""type"":""image""}",2020-12-02 16:13:41,2020-12-02 16:13:41,"0"
16,Jus de fruits bio,jus-de-fruits-bio,Délicieux jus de fruits frais sans sucre ajouté,Dakar,"0",11,Industrie alimentaire,1,11,"{""path"":""uploads\/offres\/ciHsYHVAUrKsNni6jysoRe2pX.jpg"",""type"":""image""}",2020-12-02 16:16:34,2020-12-02 16:16:34,"0"
17,Jolis sacs personnalisés 2 pour le prix d'1,jolis-sacs-personnalises-2-pour-le-prix-d1,Sacs personnalisés en promo. Stock limité,Dakar,"0",11,"Mode, industrie textile",1,11,"{""path"":""uploads\/offres\/EUo7oK5NFih4Or1BRrxLquKwk.jpg"",""type"":""image""}",2020-12-02 16:20:17,2020-12-02 16:20:17,"0"
22,Praesent lobortis velit aliquam arcu pulvinar luctus 1,praesent-lobortis-velit-aliquam-arcu-pulvinar-luctus-1,Praesent lobortis velit aliquam arcu pulvinar luctus 1,Dakar,"0",2,Automobile,1,2,"{""path"":""uploads\/offres\/OVDYJwTMqD0GJshHBVCgTOveW.jpg"",""type"":""image""}",2021-02-12 14:27:02,2021-02-12 14:27:02,"0"
24,Praesent lobortis velit aliquam arcu pulvinar luctus 2,praesent-lobortis-velit-aliquam-arcu-pulvinar-luctus-2,Praesent lobortis velit aliquam arcu pulvinar luctus 2,Dakar,"0",2,Audiovisuel - Spectacle,1,2,"{""path"":""uploads\/offres\/Ker1ckPLrMKvyVVrjP0cLSkyn.png"",""type"":""image""}",2021-02-12 14:28:53,2021-02-12 14:28:53,"0"
25,Praesent lobortis velit aliquam arcu pulvinar luctus 3,praesent-lobortis-velit-aliquam-arcu-pulvinar-luctus-3,Praesent lobortis velit aliquam arcu pulvinar luctus 3,Dakar,"0",2,"Edition, Journalisme",1,2,"{""path"":""uploads\/offres\/gI4kfZm1d5WybMcrbKze2gWqQ.png"",""type"":""image""}",2021-02-12 14:29:36,2021-02-12 14:29:36,"0"
{
"cells": [
{
"cell_type": "markdown",
"id": "7d5d99b9",
"metadata": {},
"source": [
"\n",
"<img align=\"center\" src=baamtu.jpeg style=\"width:150px\"> \n",
"<h1 align=\"center\"> You Know You Share </h1>\n",
"<h3 align=\"center\">Saliou FALL</h3>\n",
"\n",
"---------------------------------------------------------------------------------------------\n",
"|\n",
"<h3 align=\"center\"> DevOps Enginer / Baamtu </h3>\n",
"<h4 align=\"center\">Email : saliou.fall@baamtu.com</h4>\n",
"\n",
"------------------\n",
"\n",
"\n",
"\n",
"## Task \n",
"\n",
"In a perspective of integrating a recommendation system into a shop application module on . We have as a challenge to propose and implement an API based on python allowing in initially to make a random selection of the different product published by the users according to different parameters as input.\n",
"\n",
"## Principle\n",
"The principle is the following: implementing an API that will return a list of random indexes of the different offers according to the parameters entered.\n",
"\n",
"## Steps\n",
"\n",
"<ol>\n",
"<li> Making a API call to retrieve data from the database </li>\n",
"<li> Pre-proccessing state ( cleaning ) </li>\n",
"<li> randomSelect function takes as input the dataset of article service , the selection parameter.</li>\n",
"<li> randomSelect function will makes a groupby according to the parameter</li>\n",
"<li> Random selection of one element on each group. </li>\n",
"<li> Remove the selected elements from the dataset</li>\n",
"<li> Restart from 4 as long as the dataset is not empty.</li>\n",
"<li> Return the index list. </li>\n",
"</ol>\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "981cc565-b285-452f-9255-37bb2ad65a49",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: flask-mysql in /opt/conda/lib/python3.9/site-packages (from -r requirements.txt (line 1)) (1.5.2)\n",
"Requirement already satisfied: pymysql in /opt/conda/lib/python3.9/site-packages (from -r requirements.txt (line 2)) (1.0.2)\n",
"Requirement already satisfied: werkzeug in /opt/conda/lib/python3.9/site-packages (from -r requirements.txt (line 3)) (2.0.3)\n",
"Requirement already satisfied: flask_httpauth in /opt/conda/lib/python3.9/site-packages (from -r requirements.txt (line 4)) (4.5.0)\n",
"Requirement already satisfied: numpy in /opt/conda/lib/python3.9/site-packages (from -r requirements.txt (line 5)) (1.21.5)\n",
"Requirement already satisfied: requests in /opt/conda/lib/python3.9/site-packages (from -r requirements.txt (line 6)) (2.27.1)\n",
"Requirement already satisfied: fsspec in /opt/conda/lib/python3.9/site-packages (from -r requirements.txt (line 7)) (2022.2.0)\n",
"Requirement already satisfied: python-dotenv in /opt/conda/lib/python3.9/site-packages (from -r requirements.txt (line 8)) (0.19.2)\n",
"Requirement already satisfied: gunicorn in /opt/conda/lib/python3.9/site-packages (from -r requirements.txt (line 9)) (20.1.0)\n",
"Requirement already satisfied: Flask in /opt/conda/lib/python3.9/site-packages (from flask-mysql->-r requirements.txt (line 1)) (2.0.3)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.9/site-packages (from requests->-r requirements.txt (line 6)) (2021.10.8)\n",
"Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.9/site-packages (from requests->-r requirements.txt (line 6)) (1.26.8)\n",
"Requirement already satisfied: charset-normalizer~=2.0.0 in /opt/conda/lib/python3.9/site-packages (from requests->-r requirements.txt (line 6)) (2.0.12)\n",
"Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.9/site-packages (from requests->-r requirements.txt (line 6)) (3.3)\n",
"Requirement already satisfied: setuptools>=3.0 in /opt/conda/lib/python3.9/site-packages (from gunicorn->-r requirements.txt (line 9)) (60.9.3)\n",
"Requirement already satisfied: click>=7.1.2 in /opt/conda/lib/python3.9/site-packages (from Flask->flask-mysql->-r requirements.txt (line 1)) (8.0.4)\n",
"Requirement already satisfied: Jinja2>=3.0 in /opt/conda/lib/python3.9/site-packages (from Flask->flask-mysql->-r requirements.txt (line 1)) (3.0.3)\n",
"Requirement already satisfied: itsdangerous>=2.0 in /opt/conda/lib/python3.9/site-packages (from Flask->flask-mysql->-r requirements.txt (line 1)) (2.1.1)\n",
"Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.9/site-packages (from Jinja2>=3.0->Flask->flask-mysql->-r requirements.txt (line 1)) (2.1.1)\n",
"Note: you may need to restart the kernel to use updated packages.\n"
]
}
],
"source": [
"pip install -r requirements.txt"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "9b14faed",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>title</th>\n",
" <th>localite</th>\n",
" <th>category_id</th>\n",
" <th>entreprise_id</th>\n",
" <th>secteur</th>\n",
" <th>user_id</th>\n",
" <th>created_at</th>\n",
" </tr>\n",
" <tr>\n",
" <th>id</th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" <th></th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>Maecenas sit amet mi nec magna porta consectet...</td>\n",
" <td>Dakar</td>\n",
" <td>7</td>\n",
" <td>1</td>\n",
" <td>Informatique et télécoms</td>\n",
" <td>1</td>\n",
" <td>2020-08-26 15:54:36</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>Maecenas eget pharetra nulla, eget tempus nibh</td>\n",
" <td>Dakar</td>\n",
" <td>9</td>\n",
" <td>1</td>\n",
" <td>Communication - Marketing - Pub</td>\n",
" <td>1</td>\n",
" <td>2020-08-26 16:20:03</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>Eu gravida orci suscipit. Donec ac nibh ac jus...</td>\n",
" <td>Dakar</td>\n",
" <td>8</td>\n",
" <td>2</td>\n",
" <td>Informatique et télécoms</td>\n",
" <td>2</td>\n",
" <td>2020-08-31 12:44:32</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>Quisque ornare sollicitudin libero eget pulvin...</td>\n",
" <td>Dakar</td>\n",
" <td>9</td>\n",
" <td>2</td>\n",
" <td>Informatique et télécoms</td>\n",
" <td>2</td>\n",
" <td>2020-09-08 10:39:35</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>Cotizel</td>\n",
" <td>Dakar</td>\n",
" <td>7</td>\n",
" <td>5</td>\n",
" <td>Communication - Marketing - Pub</td>\n",
" <td>5</td>\n",
" <td>2020-10-23 11:10:55</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6</th>\n",
" <td>Services d'assurance</td>\n",
" <td>Kaolack</td>\n",
" <td>9</td>\n",
" <td>10</td>\n",
" <td>Banque, assurance</td>\n",
" <td>10</td>\n",
" <td>2020-10-28 12:06:43</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>ouyouma</td>\n",
" <td>Dakar</td>\n",
" <td>7</td>\n",
" <td>5</td>\n",
" <td>Social</td>\n",
" <td>5</td>\n",
" <td>2020-10-28 12:08:43</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>hdhfd</td>\n",
" <td>Dakar</td>\n",
" <td>7</td>\n",
" <td>5</td>\n",
" <td>Audit, gestion</td>\n",
" <td>5</td>\n",
" <td>2020-10-28 12:10:23</td>\n",
" </tr>\n",
" <tr>\n",
" <th>9</th>\n",
" <td>ouyouma</td>\n",
" <td>Dakar</td>\n",
" <td>7</td>\n",
" <td>5</td>\n",
" <td>Social</td>\n",
" <td>5</td>\n",
" <td>2020-11-10 12:22:40</td>\n",
" </tr>\n",
" <tr>\n",
" <th>10</th>\n",
" <td>accompagnement des pme de EVA</td>\n",
" <td>Dakar</td>\n",
" <td>0</td>\n",
" <td>5</td>\n",
" <td>Informatique et télécoms</td>\n",
" <td>5</td>\n",
" <td>2020-11-11 12:37:38</td>\n",
" </tr>\n",
" <tr>\n",
" <th>11</th>\n",
" <td>accompagnement des pme</td>\n",
" <td>Dakar</td>\n",
" <td>0</td>\n",
" <td>5</td>\n",
" <td>Audit, gestion</td>\n",
" <td>5</td>\n",
" <td>2020-11-30 17:25:15</td>\n",
" </tr>\n",
" <tr>\n",
" <th>12</th>\n",
" <td>Chaussures en cuir</td>\n",
" <td>Dakar</td>\n",
" <td>0</td>\n",
" <td>10</td>\n",
" <td>Mode, industrie textile</td>\n",
" <td>10</td>\n",
" <td>2020-12-02 11:29:13</td>\n",
" </tr>\n",
" <tr>\n",
" <th>13</th>\n",
" <td>Pagnes WAX</td>\n",
" <td>Thiès</td>\n",
" <td>0</td>\n",
" <td>10</td>\n",
" <td>Mode, industrie textile</td>\n",
" <td>10</td>\n",
" <td>2020-12-02 11:35:21</td>\n",
" </tr>\n",
" <tr>\n",
" <th>14</th>\n",
" <td>Travaux BTP</td>\n",
" <td>Fatick</td>\n",
" <td>0</td>\n",
" <td>10</td>\n",
" <td>BTP, architecture</td>\n",
" <td>10</td>\n",
" <td>2020-12-02 12:26:07</td>\n",
" </tr>\n",
" <tr>\n",
" <th>15</th>\n",
" <td>Produits capillaires</td>\n",
" <td>Dakar</td>\n",
" <td>0</td>\n",
" <td>11</td>\n",
" <td>Chimie, pharmacie</td>\n",
" <td>11</td>\n",
" <td>2020-12-02 16:13:41</td>\n",
" </tr>\n",
" <tr>\n",
" <th>16</th>\n",
" <td>Jus de fruits bio</td>\n",
" <td>Dakar</td>\n",
" <td>0</td>\n",
" <td>11</td>\n",
" <td>Industrie alimentaire</td>\n",
" <td>11</td>\n",
" <td>2020-12-02 16:16:34</td>\n",
" </tr>\n",
" <tr>\n",
" <th>17</th>\n",
" <td>Jolis sacs personnalisés 2 pour le prix d'1</td>\n",
" <td>Dakar</td>\n",
" <td>0</td>\n",
" <td>11</td>\n",
" <td>Mode, industrie textile</td>\n",
" <td>11</td>\n",
" <td>2020-12-02 16:20:17</td>\n",
" </tr>\n",
" <tr>\n",
" <th>22</th>\n",
" <td>Praesent lobortis velit aliquam arcu pulvinar ...</td>\n",
" <td>Dakar</td>\n",
" <td>0</td>\n",
" <td>2</td>\n",
" <td>Automobile</td>\n",
" <td>2</td>\n",
" <td>2021-02-12 14:27:02</td>\n",
" </tr>\n",
" <tr>\n",
" <th>24</th>\n",
" <td>Praesent lobortis velit aliquam arcu pulvinar ...</td>\n",
" <td>Dakar</td>\n",
" <td>0</td>\n",
" <td>2</td>\n",
" <td>Audiovisuel - Spectacle</td>\n",
" <td>2</td>\n",
" <td>2021-02-12 14:28:53</td>\n",
" </tr>\n",
" <tr>\n",
" <th>25</th>\n",
" <td>Praesent lobortis velit aliquam arcu pulvinar ...</td>\n",
" <td>Dakar</td>\n",
" <td>0</td>\n",
" <td>2</td>\n",
" <td>Edition, Journalisme</td>\n",
" <td>2</td>\n",
" <td>2021-02-12 14:29:36</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" title localite category_id \\\n",
"id \n",
"1 Maecenas sit amet mi nec magna porta consectet... Dakar 7 \n",
"2 Maecenas eget pharetra nulla, eget tempus nibh Dakar 9 \n",
"3 Eu gravida orci suscipit. Donec ac nibh ac jus... Dakar 8 \n",
"4 Quisque ornare sollicitudin libero eget pulvin... Dakar 9 \n",
"5 Cotizel Dakar 7 \n",
"6 Services d'assurance Kaolack 9 \n",
"7 ouyouma Dakar 7 \n",
"8 hdhfd Dakar 7 \n",
"9 ouyouma Dakar 7 \n",
"10 accompagnement des pme de EVA Dakar 0 \n",
"11 accompagnement des pme Dakar 0 \n",
"12 Chaussures en cuir Dakar 0 \n",
"13 Pagnes WAX Thiès 0 \n",
"14 Travaux BTP Fatick 0 \n",
"15 Produits capillaires Dakar 0 \n",
"16 Jus de fruits bio Dakar 0 \n",
"17 Jolis sacs personnalisés 2 pour le prix d'1 Dakar 0 \n",
"22 Praesent lobortis velit aliquam arcu pulvinar ... Dakar 0 \n",
"24 Praesent lobortis velit aliquam arcu pulvinar ... Dakar 0 \n",
"25 Praesent lobortis velit aliquam arcu pulvinar ... Dakar 0 \n",
"\n",
" entreprise_id secteur user_id \\\n",
"id \n",
"1 1 Informatique et télécoms 1 \n",
"2 1 Communication - Marketing - Pub 1 \n",
"3 2 Informatique et télécoms 2 \n",
"4 2 Informatique et télécoms 2 \n",
"5 5 Communication - Marketing - Pub 5 \n",
"6 10 Banque, assurance 10 \n",
"7 5 Social 5 \n",
"8 5 Audit, gestion 5 \n",
"9 5 Social 5 \n",
"10 5 Informatique et télécoms 5 \n",
"11 5 Audit, gestion 5 \n",
"12 10 Mode, industrie textile 10 \n",
"13 10 Mode, industrie textile 10 \n",
"14 10 BTP, architecture 10 \n",
"15 11 Chimie, pharmacie 11 \n",
"16 11 Industrie alimentaire 11 \n",
"17 11 Mode, industrie textile 11 \n",
"22 2 Automobile 2 \n",
"24 2 Audiovisuel - Spectacle 2 \n",
"25 2 Edition, Journalisme 2 \n",
"\n",
" created_at \n",
"id \n",
"1 2020-08-26 15:54:36 \n",
"2 2020-08-26 16:20:03 \n",
"3 2020-08-31 12:44:32 \n",
"4 2020-09-08 10:39:35 \n",
"5 2020-10-23 11:10:55 \n",
"6 2020-10-28 12:06:43 \n",
"7 2020-10-28 12:08:43 \n",
"8 2020-10-28 12:10:23 \n",
"9 2020-11-10 12:22:40 \n",
"10 2020-11-11 12:37:38 \n",
"11 2020-11-30 17:25:15 \n",
"12 2020-12-02 11:29:13 \n",
"13 2020-12-02 11:35:21 \n",
"14 2020-12-02 12:26:07 \n",
"15 2020-12-02 16:13:41 \n",
"16 2020-12-02 16:16:34 \n",
"17 2020-12-02 16:20:17 \n",
"22 2021-02-12 14:27:02 \n",
"24 2021-02-12 14:28:53 \n",
"25 2021-02-12 14:29:36 "
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#import the pandas library\n",
"import pandas as pd\n",
"import numpy as np\n",
"import functools\n",
"import operator\n",
"import requests\n",
"import json \n",
"\n",
"# from requests.structures import CaseInsensitiveDict\n",
"\n",
"# data = pd.read_json( json.dumps(requests.get(url, headers=headers).json())).set_index(\"id\")\n",
"# df = pd.DataFrame(data)\n",
"\n",
"df=pd.read_csv('articles.csv')\n",
"data=df.drop(columns=['slug', 'descriptif', 'attachment','updated_at','status','vues'])\n",
"data=data.set_index(\"id\")\n",
"data"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "2d799667",
"metadata": {},
"outputs": [],
"source": [
"def randomSelect(data,group ,size, replace):\n",
" index=[]\n",
" fn = lambda obj: obj.loc[np.random.choice(obj.index, 1 , replace),:]\n",
" while data.empty != True:\n",
" randomListByUser=data.groupby(group, as_index=False).apply(fn)\n",
" index.append( [ item for item in randomListByUser.index.get_level_values(\"id\") ] )\n",
" data=data.drop(randomListByUser.index.get_level_values(\"id\"), axis=0)\n",
" return functools.reduce(operator.iconcat, index, []) "
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "a851a487",
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/plain": [
"'[2, 3, 7, 12, 17, 1, 25, 11, 13, 15, 4, 8, 6, 16, 22, 9, 14, 24, 10, 5]'"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"\n",
"size = 1 # sample size\n",
"replace = False # with replacement\n",
"group='entreprise_id'\n",
"json.dumps(randomSelect(data, group, size, replace))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cf9d7039",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.10"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
id,title,slug,descriptif,localite,category_id,entreprise_id,secteur,status,user_id,attachment,created_at,updated_at,vues
1,Maecenas sit amet mi nec magna porta consectetur eu ac odio,maecenas-sit-amet-mi-nec-magna-porta-consectetur-eu-ac-odio,"<p><span style=""color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif; text-align: justify;"">Duis pretium diam a felis pretium, a rutrum nibh pretium. Vestibulum porta rutrum turpis. Maecenas sit amet mi nec magna porta consectetur eu ac odio. Phasellus quis justo gravida justo tincidunt interdum. Aenean tristique erat varius ante efficitur imperdiet. Vivamus magna massa, luctus id congue in, sollicitudin at sem. Pellentesque cursus vestibulum purus, vitae elementum nisi aliquam in. Phasellus scelerisque, justo eu molestie ultrices, ipsum lacus condimentum orci, id porttitor orci nisl at libero. Nullam semper lorem nisi, aliquam consequat odio fringilla et.&nbsp;</span></p><p><span style=""color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif; text-align: justify;"">Etiam id turpis vel dolor ornare lacinia. Quisque pharetra, enim ut laoreet luctus, risus metus congue nibh, faucibus fermentum lectus ex a nisi. Donec tristique libero consequat lorem efficitur, vitae tristique felis varius. Fusce tristique augue nec risus convallis, vitae efficitur est varius. Morbi tempor leo non rutrum euismod. Donec iaculis nisi quis purus feugiat auctor.</span><br></p>",Dakar,7,1,Informatique et télécoms,1,1,"{""path"": ""uploads/offres/qMQhmmdrOeDWbcZE7enmZJHMx.jpg"", ""type"": ""image""}",2020-08-26 15:54:36,2020-09-08 16:09:59,"0"
2,"Maecenas eget pharetra nulla, eget tempus nibh",maecenas-eget-pharetra-nulla-eget-tempus-nibh,"<p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px; text-align: justify; color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif;"">Quisque nulla quam, pulvinar sit amet luctus eu, rhoncus ac ex. Sed dapibus bibendum magna vulputate mollis. Curabitur non urna blandit, congue tellus eu, feugiat ante.&nbsp;</p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px; text-align: justify; color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif;""><ul><li>Nulla quis tellus vitae massa fringilla sagittis.&nbsp;</li><li>Vivamus sapien velit, consequat</li><li>&nbsp;maximus at, finibus pharetra purus.</li></ul></p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px; text-align: justify; color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif;"">&nbsp;Pellentesque id turpis in mi aliquet pulvinar. Quisque rhoncus orci et elementum pulvinar. Donec scelerisque eros eget tristique tempus. Proin facilisis orci in enim porttitor, nec pulvinar urna vulputate. Morbi fringilla massa id sapien gravida, eu vestibulum sem bibendum. Nulla eleifend viverra enim non sodales. Phasellus maximus placerat dolor nec luctus. Sed tincidunt mauris ligula, non accumsan nunc laoreet sed. Maecenas eget pharetra nulla, eget tempus nibh.</p><p><br></p>",Dakar,9,1,Communication - Marketing - Pub,1,1,"{""path"": ""uploads/offres/LmJ6933bCP7tQM4405leZuh9r.jpg"", ""type"": ""image""}",2020-08-26 16:20:03,2020-08-26 16:20:03,"0"
3,Eu gravida orci suscipit. Donec ac nibh ac justo rutrum tempus,eu-gravida-orci-suscipit-donec-ac-nibh-ac-justo-rutrum-tempus,"<p id=""lipsum"" style=""margin: 0px; padding: 0px; text-align: justify;""></p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">In hac habitasse platea dictumst. Aliquam ac dui odio. Etiam euismod sit amet tortor eget tincidunt. Sed eu ante dapibus ipsum pellentesque congue id sit amet arcu. Phasellus eget risus consectetur, imperdiet lorem eu, sollicitudin diam. Nam eu enim leo. Fusce ultrices dictum ligula auctor auctor. Morbi vestibulum tristique quam nec imperdiet.</p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">Donec sagittis dui at facilisis mattis. Maecenas et lacus eros. Nulla vehicula elit diam, eget euismod nisl aliquam nec. Duis suscipit sapien nec enim consectetur, quis rutrum sapien consectetur. Nulla in gravida nisl, at fringilla urna. Maecenas at ornare nisl. Nam lacinia faucibus urna, in ultrices lacus sagittis et. Pellentesque hendrerit vehicula lorem, at ultricies lacus. Aliquam dictum magna vitae erat ornare, quis commodo ante dapibus. Nunc eget blandit ipsum. Quisque congue massa a diam imperdiet fermentum. Aliquam pulvinar vehicula arcu, non suscipit metus pulvinar vitae.</p><p style=""color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif;""><br></p><p></p>",Dakar,8,2,Informatique et télécoms,1,2,"{""path"": ""uploads/offres/UyBMrfL8Wj2GnQ82wJf70aJww.jpg"", ""type"": ""image""}",2020-08-31 12:44:32,2020-08-31 12:44:32,"0"
4,Quisque ornare sollicitudin libero eget pulvinar pellentesque convallis neque sit amet urna vestibulum commodo,quisque-ornare-sollicitudin-libero-eget-pulvinar-pellentesque-convallis-neque-sit-amet-urna-vestibulum-commodo,"<p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">Sed malesuada, ante sit amet ultrices iaculis, est justo scelerisque nisi, quis vehicula nibh lorem et dolor. Nunc ut elementum mauris. Morbi in erat at nibh convallis efficitur. Ut blandit urna quis eleifend auctor. Proin laoreet sagittis urna non consequat. Quisque viverra cursus ullamcorper. Ut accumsan sem at aliquet bibendum. Cras molestie tempus metus, ac efficitur lorem porta vitae.</p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">Donec vitae sagittis felis, et fermentum dolor. Nullam eleifend metus nec ante malesuada finibus. Nulla varius egestas ante, ut ullamcorper ante suscipit ac. Quisque ornare posuere dictum. Morbi dictum velit sed sapien fringilla, at viverra tellus malesuada. In tincidunt felis ut mauris feugiat, a auctor risus accumsan. Mauris faucibus odio lacus, quis lacinia nulla tincidunt at. Praesent ligula justo, pretium sed nisi in, hendrerit congue enim. Fusce volutpat quam libero, at rhoncus dui laoreet ut. Phasellus porta pretium augue, et consectetur nisl consequat condimentum. Suspendisse potenti. Sed ornare vulputate ante, semper ornare tellus iaculis et. Proin lacinia laoreet eros sit amet congue. Aenean euismod tristique lobortis. Praesent quis ex at nisi egestas elementum. Suspendisse efficitur dapibus est, eu lobortis mi mollis ut.</p><p style=""margin-right: 0px; margin-bottom: 15px; margin-left: 0px; padding: 0px;"">In interdum eros ante, eget mattis mi vulputate sed. Nulla interdum viverra accumsan. Quisque vel leo posuere neque sagittis aliquet. Aenean porta magna eu erat dictum, feugiat cursus ex iaculis. Sed nec hendrerit nulla. Suspendisse potenti. Ut eu condimentum nulla. Duis non eros eu arcu tincidunt ullamcorper non a ligula. Suspendisse dictum faucibus ex, ac volutpat libero auctor in. Etiam scelerisque dolor vehicula velit pellentesque elementum.</p><p><br></p>",Dakar,9,2,Informatique et télécoms,1,2,"{""path"": ""uploads/offres/rTuVxWTg7ArfBjyMonns3o6cr.png"", ""type"": ""image""}",2020-09-08 10:39:35,2020-09-08 10:39:35,"0"
5,Cotizel,cotizel,"Cotizel est une solution digital d'encaissement via mobile money.
A travers son CRM et son système d'integration, elle permet aux opérateurs économiques d'encaisser des flux de paiements, uniques ou récurrent via mobile money ou carte bancaire, sans contraintes technologiques.",Dakar,7,5,Communication - Marketing - Pub,1,5,"{""path"":""uploads\/offres\/4AKSMLept5zxZjMufAYmBV1VQ.png"",""type"":""image""}",2020-10-23 11:10:55,2020-12-01 18:53:17,"0"
6,Services d'assurance,services-dassurance,Venez découvrir nos pack d'assurance à partir de 10 000FCFA,Kaolack,9,10,"Banque, assurance",1,10,"{""path"":""uploads\/offres\/yYqVYkAj215sXu7ElN0Bi4SLC.jpg"",""type"":""image""}",2020-10-28 12:06:43,2020-12-02 12:31:59,"0"
7,ouyouma,ouyouma,Accompagnement psychosocial pour les personnes physiques en détresse émotionnelle.,Dakar,7,5,Social,1,5,"{""path"":""uploads\/offres\/Itd5W7DomJ8BMVS8ITbPq9ZK4.png"",""type"":""image""}",2020-10-28 12:08:43,2020-10-28 12:08:51,"0"
8,hdhfd,hdhfd,hgfjdshndyjhnhkj,Dakar,7,5,"Audit, gestion",1,5,"{""path"":""uploads\/offres\/DV8YGpQWxbaIapC6mJipMSOew.jpg"",""type"":""image""}",2020-10-28 12:10:23,2020-10-28 12:12:28,"0"
9,ouyouma,ouyouma,accompagnement psychologique pour les pme,Dakar,7,5,Social,10,5,,2020-11-10 12:22:40,2020-11-10 12:30:40,"0"
10,accompagnement des pme de EVA,accompagnement-des-pme-de-eva,nous accompagnons les pme a se digitaliser de façon soft prenant en compte leur contrainte de temps et financière.,Dakar,"0",5,Informatique et télécoms,1,5,,2020-11-11 12:37:38,2020-11-11 12:37:38,"0"
11,accompagnement des pme,accompagnement-des-pme,Nous accompagnos les PME,Dakar,"0",5,"Audit, gestion",1,5,"{""path"":""uploads\/offres\/Moml4MwKRtnrIXJ3mr3s8jzcd.png"",""type"":""image""}",2020-11-30 17:25:15,2020-11-30 17:25:15,"0"
12,Chaussures en cuir,chaussures-en-cuir,Venez découvrir nos chaussures en cuir de Ngay Mékhé,Dakar,"0",10,"Mode, industrie textile",1,10,"{""path"":""uploads\/offres\/EW0kVdBHfyIqjv8iQTsJ2lQAj.jpg"",""type"":""image""}",2020-12-02 11:29:13,2020-12-02 12:29:20,"0"
13,Pagnes WAX,pagnes-wax,Superbes Pagnes WAX en promotion,Thiès,"0",10,"Mode, industrie textile",1,10,"{""path"":""uploads\/offres\/OADnCJ29QYrrPhDI5usBoV0Gj.jpg"",""type"":""image""}",2020-12-02 11:35:21,2020-12-02 12:28:49,"0"
14,Travaux BTP,travaux-btp,Bénéficiez des meilleures services BTP avec un tarif adapté,Fatick,"0",10,"BTP, architecture",1,10,"{""path"":""uploads\/offres\/CtqSfbLMMpeBI9U2mei36FdRn.jpg"",""type"":""image""}",2020-12-02 12:26:07,2020-12-02 12:26:07,"0"
15,Produits capillaires,produits-capillaires,Vente de produits capillaires sans sulfate,Dakar,"0",11,"Chimie, pharmacie",1,11,"{""path"":""uploads\/offres\/3UGRUD5Cg7THxSV59ebrcK7ut.jpg"",""type"":""image""}",2020-12-02 16:13:41,2020-12-02 16:13:41,"0"
16,Jus de fruits bio,jus-de-fruits-bio,Délicieux jus de fruits frais sans sucre ajouté,Dakar,"0",11,Industrie alimentaire,1,11,"{""path"":""uploads\/offres\/ciHsYHVAUrKsNni6jysoRe2pX.jpg"",""type"":""image""}",2020-12-02 16:16:34,2020-12-02 16:16:34,"0"
17,Jolis sacs personnalisés 2 pour le prix d'1,jolis-sacs-personnalises-2-pour-le-prix-d1,Sacs personnalisés en promo. Stock limité,Dakar,"0",11,"Mode, industrie textile",1,11,"{""path"":""uploads\/offres\/EUo7oK5NFih4Or1BRrxLquKwk.jpg"",""type"":""image""}",2020-12-02 16:20:17,2020-12-02 16:20:17,"0"
22,Praesent lobortis velit aliquam arcu pulvinar luctus 1,praesent-lobortis-velit-aliquam-arcu-pulvinar-luctus-1,Praesent lobortis velit aliquam arcu pulvinar luctus 1,Dakar,"0",2,Automobile,1,2,"{""path"":""uploads\/offres\/OVDYJwTMqD0GJshHBVCgTOveW.jpg"",""type"":""image""}",2021-02-12 14:27:02,2021-02-12 14:27:02,"0"
24,Praesent lobortis velit aliquam arcu pulvinar luctus 2,praesent-lobortis-velit-aliquam-arcu-pulvinar-luctus-2,Praesent lobortis velit aliquam arcu pulvinar luctus 2,Dakar,"0",2,Audiovisuel - Spectacle,1,2,"{""path"":""uploads\/offres\/Ker1ckPLrMKvyVVrjP0cLSkyn.png"",""type"":""image""}",2021-02-12 14:28:53,2021-02-12 14:28:53,"0"
25,Praesent lobortis velit aliquam arcu pulvinar luctus 3,praesent-lobortis-velit-aliquam-arcu-pulvinar-luctus-3,Praesent lobortis velit aliquam arcu pulvinar luctus 3,Dakar,"0",2,"Edition, Journalisme",1,2,"{""path"":""uploads\/offres\/gI4kfZm1d5WybMcrbKze2gWqQ.png"",""type"":""image""}",2021-02-12 14:29:36,2021-02-12 14:29:36,"0"
version: "3"
services:
datascience-notebook:
image: jupyter/datascience-notebook
volumes:
- ./:/home/jovyan/work
ports:
- 8888:8888
container_name: datascience-notebook-container
flask-mysql
pymysql
werkzeug
flask_httpauth
numpy
requests
fsspec
python-dotenv
gunicorn
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment